2023-08-23 10:53:10,866 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-08-23 10:53:11,117 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc 2023-08-23 10:53:11,134 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-08-23 10:53:11,135 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a, deleteOnExit=true 2023-08-23 10:53:11,135 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-08-23 10:53:11,136 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/test.cache.data in system properties and HBase conf 2023-08-23 10:53:11,136 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/hadoop.tmp.dir in system properties and HBase conf 2023-08-23 10:53:11,137 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/hadoop.log.dir in system properties and HBase conf 2023-08-23 10:53:11,138 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/mapreduce.cluster.local.dir in system properties and HBase conf 2023-08-23 10:53:11,138 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-08-23 10:53:11,138 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-08-23 10:53:11,252 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-08-23 10:53:11,685 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-08-23 10:53:11,692 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-08-23 10:53:11,692 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-08-23 10:53:11,693 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-08-23 10:53:11,693 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-23 10:53:11,694 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-08-23 10:53:11,694 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-08-23 10:53:11,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-08-23 10:53:11,695 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-23 10:53:11,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-08-23 10:53:11,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/nfs.dump.dir in system properties and HBase conf 2023-08-23 10:53:11,696 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/java.io.tmpdir in system properties and HBase conf 2023-08-23 10:53:11,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/dfs.journalnode.edits.dir in system properties and HBase conf 2023-08-23 10:53:11,697 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-08-23 10:53:11,698 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-08-23 10:53:12,315 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-23 10:53:12,319 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-23 10:53:12,616 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-08-23 10:53:12,795 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-08-23 10:53:12,822 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-23 10:53:12,875 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-08-23 10:53:12,919 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/java.io.tmpdir/Jetty_localhost_36387_hdfs____lxuuop/webapp 2023-08-23 10:53:13,088 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36387 2023-08-23 10:53:13,127 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-08-23 10:53:13,127 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-08-23 10:53:13,750 WARN [Listener at localhost/34277] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-23 10:53:13,839 WARN [Listener at localhost/34277] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-23 10:53:13,865 WARN [Listener at localhost/34277] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-23 10:53:13,874 INFO [Listener at localhost/34277] log.Slf4jLog(67): jetty-6.1.26 2023-08-23 10:53:13,880 INFO [Listener at localhost/34277] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/java.io.tmpdir/Jetty_localhost_42623_datanode____.rhth40/webapp 2023-08-23 10:53:13,981 INFO [Listener at localhost/34277] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42623 2023-08-23 10:53:14,310 WARN [Listener at localhost/37963] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-23 10:53:14,330 WARN [Listener at localhost/37963] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-23 10:53:14,337 WARN [Listener at localhost/37963] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-23 10:53:14,339 INFO [Listener at localhost/37963] log.Slf4jLog(67): jetty-6.1.26 2023-08-23 10:53:14,347 INFO [Listener at localhost/37963] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/java.io.tmpdir/Jetty_localhost_44427_datanode____semvs0/webapp 2023-08-23 10:53:14,490 INFO [Listener at localhost/37963] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44427 2023-08-23 10:53:14,513 WARN [Listener at localhost/41061] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-23 10:53:14,584 WARN [Listener at localhost/41061] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-08-23 10:53:14,589 WARN [Listener at localhost/41061] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-08-23 10:53:14,591 INFO [Listener at localhost/41061] log.Slf4jLog(67): jetty-6.1.26 2023-08-23 10:53:14,603 INFO [Listener at localhost/41061] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/java.io.tmpdir/Jetty_localhost_46259_datanode____.23n5hh/webapp 2023-08-23 10:53:14,759 INFO [Listener at localhost/41061] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46259 2023-08-23 10:53:14,797 WARN [Listener at localhost/34585] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-08-23 10:53:15,002 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x448a8c740320918b: Processing first storage report for DS-718d104c-374b-40e2-b5ff-b6546bb4eddf from datanode 44620c13-9adc-474d-8c34-bedbb2aa5c0d 2023-08-23 10:53:15,004 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x448a8c740320918b: from storage DS-718d104c-374b-40e2-b5ff-b6546bb4eddf node DatanodeRegistration(127.0.0.1:45527, datanodeUuid=44620c13-9adc-474d-8c34-bedbb2aa5c0d, infoPort=45375, infoSecurePort=0, ipcPort=34585, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,004 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3ff57206be3dc710: Processing first storage report for DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775 from datanode 0db35eca-a409-482d-977c-53ac02140840 2023-08-23 10:53:15,004 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3ff57206be3dc710: from storage DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775 node DatanodeRegistration(127.0.0.1:44203, datanodeUuid=0db35eca-a409-482d-977c-53ac02140840, infoPort=38033, infoSecurePort=0, ipcPort=37963, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,005 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd81e0a01c0610266: Processing first storage report for DS-dc374cc7-e08a-46e7-affa-82501381bb96 from datanode 286f5afc-c3f2-4f9a-8396-78cfba18066f 2023-08-23 10:53:15,005 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd81e0a01c0610266: from storage DS-dc374cc7-e08a-46e7-affa-82501381bb96 node DatanodeRegistration(127.0.0.1:41707, datanodeUuid=286f5afc-c3f2-4f9a-8396-78cfba18066f, infoPort=45193, infoSecurePort=0, ipcPort=41061, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,005 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x448a8c740320918b: Processing first storage report for DS-bc45a4d7-442a-4dd2-957d-07d4fccd1406 from datanode 44620c13-9adc-474d-8c34-bedbb2aa5c0d 2023-08-23 10:53:15,005 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x448a8c740320918b: from storage DS-bc45a4d7-442a-4dd2-957d-07d4fccd1406 node DatanodeRegistration(127.0.0.1:45527, datanodeUuid=44620c13-9adc-474d-8c34-bedbb2aa5c0d, infoPort=45375, infoSecurePort=0, ipcPort=34585, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,005 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3ff57206be3dc710: Processing first storage report for DS-e773ef86-2548-4c5b-a9c1-2c48d4ef65e3 from datanode 0db35eca-a409-482d-977c-53ac02140840 2023-08-23 10:53:15,006 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3ff57206be3dc710: from storage DS-e773ef86-2548-4c5b-a9c1-2c48d4ef65e3 node DatanodeRegistration(127.0.0.1:44203, datanodeUuid=0db35eca-a409-482d-977c-53ac02140840, infoPort=38033, infoSecurePort=0, ipcPort=37963, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,006 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd81e0a01c0610266: Processing first storage report for DS-010aad26-1b6a-4b1c-9710-91db7f74da03 from datanode 286f5afc-c3f2-4f9a-8396-78cfba18066f 2023-08-23 10:53:15,006 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd81e0a01c0610266: from storage DS-010aad26-1b6a-4b1c-9710-91db7f74da03 node DatanodeRegistration(127.0.0.1:41707, datanodeUuid=286f5afc-c3f2-4f9a-8396-78cfba18066f, infoPort=45193, infoSecurePort=0, ipcPort=41061, storageInfo=lv=-57;cid=testClusterID;nsid=824207077;c=1692787992396), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-08-23 10:53:15,224 DEBUG [Listener at localhost/34585] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc 2023-08-23 10:53:15,299 INFO [Listener at localhost/34585] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/zookeeper_0, clientPort=52557, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-08-23 10:53:15,314 INFO [Listener at localhost/34585] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52557 2023-08-23 10:53:15,327 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:15,330 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:16,066 INFO [Listener at localhost/34585] util.FSUtils(471): Created version file at hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e with version=8 2023-08-23 10:53:16,066 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/hbase-staging 2023-08-23 10:53:16,426 INFO [Listener at localhost/34585] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-08-23 10:53:16,884 INFO [Listener at localhost/34585] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-08-23 10:53:16,916 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:16,917 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:16,917 INFO [Listener at localhost/34585] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-23 10:53:16,918 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:16,918 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-23 10:53:17,058 INFO [Listener at localhost/34585] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-08-23 10:53:17,129 DEBUG [Listener at localhost/34585] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-08-23 10:53:17,224 INFO [Listener at localhost/34585] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44301 2023-08-23 10:53:17,234 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,236 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,257 INFO [Listener at localhost/34585] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44301 connecting to ZooKeeper ensemble=127.0.0.1:52557 2023-08-23 10:53:17,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:443010x0, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-23 10:53:17,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44301-0x10038c62a350000 connected 2023-08-23 10:53:17,340 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-23 10:53:17,341 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:53:17,345 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-23 10:53:17,357 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44301 2023-08-23 10:53:17,357 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44301 2023-08-23 10:53:17,358 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44301 2023-08-23 10:53:17,363 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44301 2023-08-23 10:53:17,363 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44301 2023-08-23 10:53:17,369 INFO [Listener at localhost/34585] master.HMaster(444): hbase.rootdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e, hbase.cluster.distributed=false 2023-08-23 10:53:17,439 INFO [Listener at localhost/34585] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-23 10:53:17,439 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,440 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,440 INFO [Listener at localhost/34585] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-23 10:53:17,440 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,440 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-23 10:53:17,445 INFO [Listener at localhost/34585] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-23 10:53:17,447 INFO [Listener at localhost/34585] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37675 2023-08-23 10:53:17,450 INFO [Listener at localhost/34585] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-23 10:53:17,456 DEBUG [Listener at localhost/34585] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-23 10:53:17,457 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,459 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,461 INFO [Listener at localhost/34585] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37675 connecting to ZooKeeper ensemble=127.0.0.1:52557 2023-08-23 10:53:17,468 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:376750x0, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-23 10:53:17,469 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37675-0x10038c62a350001 connected 2023-08-23 10:53:17,469 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-23 10:53:17,471 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:53:17,472 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-23 10:53:17,472 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37675 2023-08-23 10:53:17,473 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37675 2023-08-23 10:53:17,473 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37675 2023-08-23 10:53:17,477 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37675 2023-08-23 10:53:17,477 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37675 2023-08-23 10:53:17,489 INFO [Listener at localhost/34585] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-23 10:53:17,490 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,490 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,490 INFO [Listener at localhost/34585] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-23 10:53:17,490 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,491 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-23 10:53:17,491 INFO [Listener at localhost/34585] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-23 10:53:17,493 INFO [Listener at localhost/34585] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41821 2023-08-23 10:53:17,493 INFO [Listener at localhost/34585] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-23 10:53:17,496 DEBUG [Listener at localhost/34585] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-23 10:53:17,497 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,500 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,502 INFO [Listener at localhost/34585] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41821 connecting to ZooKeeper ensemble=127.0.0.1:52557 2023-08-23 10:53:17,508 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:418210x0, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-23 10:53:17,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41821-0x10038c62a350002 connected 2023-08-23 10:53:17,510 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-23 10:53:17,510 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:53:17,511 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-23 10:53:17,516 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41821 2023-08-23 10:53:17,516 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41821 2023-08-23 10:53:17,518 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41821 2023-08-23 10:53:17,519 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41821 2023-08-23 10:53:17,521 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41821 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-08-23 10:53:17,533 INFO [Listener at localhost/34585] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-08-23 10:53:17,538 INFO [Listener at localhost/34585] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43499 2023-08-23 10:53:17,538 INFO [Listener at localhost/34585] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-08-23 10:53:17,540 DEBUG [Listener at localhost/34585] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-08-23 10:53:17,541 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,542 INFO [Listener at localhost/34585] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,543 INFO [Listener at localhost/34585] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43499 connecting to ZooKeeper ensemble=127.0.0.1:52557 2023-08-23 10:53:17,546 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:434990x0, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-08-23 10:53:17,547 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:434990x0, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-23 10:53:17,548 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:434990x0, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:53:17,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:43499-0x10038c62a350003 connected 2023-08-23 10:53:17,549 DEBUG [Listener at localhost/34585] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-08-23 10:53:17,553 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43499 2023-08-23 10:53:17,553 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43499 2023-08-23 10:53:17,554 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43499 2023-08-23 10:53:17,555 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43499 2023-08-23 10:53:17,561 DEBUG [Listener at localhost/34585] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43499 2023-08-23 10:53:17,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:17,572 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-23 10:53:17,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:17,600 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-23 10:53:17,600 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-23 10:53:17,600 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-23 10:53:17,600 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-08-23 10:53:17,600 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:17,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-23 10:53:17,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,44301,1692787996219 from backup master directory 2023-08-23 10:53:17,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-08-23 10:53:17,607 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:17,607 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-08-23 10:53:17,608 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-23 10:53:17,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:17,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-08-23 10:53:17,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-08-23 10:53:17,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/hbase.id with ID: a8b16e09-368f-4390-b18c-560fc7537f56 2023-08-23 10:53:17,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-08-23 10:53:17,765 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:17,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5638e7f8 to 127.0.0.1:52557 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-23 10:53:17,870 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@644928f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-23 10:53:17,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-23 10:53:17,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-08-23 10:53:17,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-08-23 10:53:17,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-08-23 10:53:17,916 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:53:17,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:53:17,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-23 10:53:17,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store-tmp 2023-08-23 10:53:18,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:18,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-23 10:53:18,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:53:18,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:53:18,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-23 10:53:18,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:53:18,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:53:18,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-23 10:53:18,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/WALs/jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:18,029 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44301%2C1692787996219, suffix=, logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/WALs/jenkins-hbase4.apache.org,44301,1692787996219, archiveDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/oldWALs, maxLogs=10 2023-08-23 10:53:18,092 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK] 2023-08-23 10:53:18,092 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK] 2023-08-23 10:53:18,092 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK] 2023-08-23 10:53:18,100 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:53:18,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/WALs/jenkins-hbase4.apache.org,44301,1692787996219/jenkins-hbase4.apache.org%2C44301%2C1692787996219.1692787998039 2023-08-23 10:53:18,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK], DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK], DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK]] 2023-08-23 10:53:18,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:18,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:18,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,247 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,254 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-08-23 10:53:18,278 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-08-23 10:53:18,289 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:18,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-08-23 10:53:18,322 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:18,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59330941, jitterRate=-0.11590008437633514}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-23 10:53:18,324 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-23 10:53:18,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-08-23 10:53:18,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:18,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:18,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:18,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-08-23 10:53:18,391 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-08-23 10:53:18,391 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-08-23 10:53:18,416 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-08-23 10:53:18,421 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-08-23 10:53:18,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-08-23 10:53:18,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-08-23 10:53:18,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-08-23 10:53:18,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-08-23 10:53:18,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-08-23 10:53:18,461 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:18,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-08-23 10:53:18,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-08-23 10:53:18,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-08-23 10:53:18,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-23 10:53:18,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-23 10:53:18,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-23 10:53:18,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-08-23 10:53:18,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:18,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44301,1692787996219, sessionid=0x10038c62a350000, setting cluster-up flag (Was=false) 2023-08-23 10:53:18,505 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:18,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-08-23 10:53:18,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:18,519 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:18,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-08-23 10:53:18,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:18,532 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.hbase-snapshot/.tmp 2023-08-23 10:53:18,566 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(951): ClusterId : a8b16e09-368f-4390-b18c-560fc7537f56 2023-08-23 10:53:18,566 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(951): ClusterId : a8b16e09-368f-4390-b18c-560fc7537f56 2023-08-23 10:53:18,567 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(951): ClusterId : a8b16e09-368f-4390-b18c-560fc7537f56 2023-08-23 10:53:18,573 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-23 10:53:18,573 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-23 10:53:18,573 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-08-23 10:53:18,580 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-23 10:53:18,580 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-23 10:53:18,580 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-08-23 10:53:18,581 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-23 10:53:18,580 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-23 10:53:18,581 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-08-23 10:53:18,589 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-23 10:53:18,589 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-23 10:53:18,589 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-08-23 10:53:18,598 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ReadOnlyZKClient(139): Connect 0x13f3c187 to 127.0.0.1:52557 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-23 10:53:18,599 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ReadOnlyZKClient(139): Connect 0x048344f7 to 127.0.0.1:52557 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-23 10:53:18,600 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ReadOnlyZKClient(139): Connect 0x71035186 to 127.0.0.1:52557 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-23 10:53:18,618 DEBUG [RS:1;jenkins-hbase4:41821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d342d89, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-23 10:53:18,620 DEBUG [RS:0;jenkins-hbase4:37675] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a44ba17, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-23 10:53:18,619 DEBUG [RS:2;jenkins-hbase4:43499] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fc4b1d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-23 10:53:18,622 DEBUG [RS:1;jenkins-hbase4:41821] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@190baaf7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-23 10:53:18,622 DEBUG [RS:0;jenkins-hbase4:37675] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e7b2f87, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-23 10:53:18,622 DEBUG [RS:2;jenkins-hbase4:43499] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bfb3af6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-23 10:53:18,656 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:43499 2023-08-23 10:53:18,661 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37675 2023-08-23 10:53:18,657 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:41821 2023-08-23 10:53:18,667 INFO [RS:1;jenkins-hbase4:41821] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-23 10:53:18,668 INFO [RS:1;jenkins-hbase4:41821] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-23 10:53:18,667 INFO [RS:2;jenkins-hbase4:43499] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-23 10:53:18,667 INFO [RS:0;jenkins-hbase4:37675] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-08-23 10:53:18,668 INFO [RS:2;jenkins-hbase4:43499] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-23 10:53:18,668 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1022): About to register with Master. 2023-08-23 10:53:18,668 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1022): About to register with Master. 2023-08-23 10:53:18,668 INFO [RS:0;jenkins-hbase4:37675] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-08-23 10:53:18,668 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1022): About to register with Master. 2023-08-23 10:53:18,672 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44301,1692787996219 with isa=jenkins-hbase4.apache.org/172.31.14.131:37675, startcode=1692787997438 2023-08-23 10:53:18,672 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44301,1692787996219 with isa=jenkins-hbase4.apache.org/172.31.14.131:41821, startcode=1692787997489 2023-08-23 10:53:18,672 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,44301,1692787996219 with isa=jenkins-hbase4.apache.org/172.31.14.131:43499, startcode=1692787997532 2023-08-23 10:53:18,691 DEBUG [RS:0;jenkins-hbase4:37675] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-23 10:53:18,691 DEBUG [RS:1;jenkins-hbase4:41821] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-23 10:53:18,691 DEBUG [RS:2;jenkins-hbase4:43499] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-08-23 10:53:18,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:18,716 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-23 10:53:18,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,720 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788028720 2023-08-23 10:53:18,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-08-23 10:53:18,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-08-23 10:53:18,737 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:18,738 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-08-23 10:53:18,745 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-23 10:53:18,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-08-23 10:53:18,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-08-23 10:53:18,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-08-23 10:53:18,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-08-23 10:53:18,747 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-08-23 10:53:18,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-08-23 10:53:18,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-08-23 10:53:18,757 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-08-23 10:53:18,757 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40127, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-08-23 10:53:18,758 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42905, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-08-23 10:53:18,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-08-23 10:53:18,758 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33983, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-08-23 10:53:18,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692787998764,5,FailOnTimeoutGroup] 2023-08-23 10:53:18,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692787998770,5,FailOnTimeoutGroup] 2023-08-23 10:53:18,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-08-23 10:53:18,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,781 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44301] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,805 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44301] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,807 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44301] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,808 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:18,809 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:18,810 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e 2023-08-23 10:53:18,813 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e 2023-08-23 10:53:18,814 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34277 2023-08-23 10:53:18,814 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-23 10:53:18,817 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e 2023-08-23 10:53:18,817 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e 2023-08-23 10:53:18,817 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34277 2023-08-23 10:53:18,817 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34277 2023-08-23 10:53:18,818 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-23 10:53:18,819 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-08-23 10:53:18,823 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:53:18,827 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ZKUtil(162): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,827 WARN [RS:2;jenkins-hbase4:43499] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-23 10:53:18,827 INFO [RS:2;jenkins-hbase4:43499] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-23 10:53:18,828 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,828 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ZKUtil(162): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,828 WARN [RS:1;jenkins-hbase4:41821] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-23 10:53:18,828 INFO [RS:1;jenkins-hbase4:41821] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-23 10:53:18,829 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ZKUtil(162): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,829 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43499,1692787997532] 2023-08-23 10:53:18,829 WARN [RS:0;jenkins-hbase4:37675] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-08-23 10:53:18,829 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,829 INFO [RS:0;jenkins-hbase4:37675] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-23 10:53:18,829 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37675,1692787997438] 2023-08-23 10:53:18,830 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41821,1692787997489] 2023-08-23 10:53:18,830 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,864 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ZKUtil(162): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,864 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ZKUtil(162): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,865 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ZKUtil(162): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,865 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ZKUtil(162): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,865 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:18,865 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ZKUtil(162): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,866 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ZKUtil(162): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,866 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ZKUtil(162): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:18,867 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ZKUtil(162): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:18,868 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-23 10:53:18,868 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ZKUtil(162): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:18,871 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/info 2023-08-23 10:53:18,872 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-23 10:53:18,874 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:18,875 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-23 10:53:18,878 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/rep_barrier 2023-08-23 10:53:18,878 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-23 10:53:18,879 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:18,879 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-23 10:53:18,881 DEBUG [RS:2;jenkins-hbase4:43499] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-23 10:53:18,881 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-23 10:53:18,881 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-08-23 10:53:18,882 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/table 2023-08-23 10:53:18,883 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-23 10:53:18,884 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:18,889 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740 2023-08-23 10:53:18,890 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740 2023-08-23 10:53:18,894 INFO [RS:1;jenkins-hbase4:41821] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-23 10:53:18,894 INFO [RS:0;jenkins-hbase4:37675] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-23 10:53:18,895 INFO [RS:2;jenkins-hbase4:43499] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-08-23 10:53:18,896 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-23 10:53:18,898 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-23 10:53:18,902 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:18,903 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72621881, jitterRate=0.08215035498142242}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-23 10:53:18,903 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-23 10:53:18,903 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-23 10:53:18,904 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-23 10:53:18,904 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-23 10:53:18,904 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-23 10:53:18,904 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-23 10:53:18,906 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-23 10:53:18,906 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-23 10:53:18,915 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:18,915 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-08-23 10:53:18,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-08-23 10:53:18,928 INFO [RS:1;jenkins-hbase4:41821] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-23 10:53:18,928 INFO [RS:0;jenkins-hbase4:37675] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-23 10:53:18,928 INFO [RS:2;jenkins-hbase4:43499] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-08-23 10:53:18,933 INFO [RS:0;jenkins-hbase4:37675] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-23 10:53:18,934 INFO [RS:2;jenkins-hbase4:43499] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-23 10:53:18,934 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,934 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,933 INFO [RS:1;jenkins-hbase4:41821] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-08-23 10:53:18,935 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,937 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-23 10:53:18,941 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-23 10:53:18,941 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-08-23 10:53:18,950 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,950 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,950 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,950 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,950 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,950 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-23 10:53:18,952 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-23 10:53:18,952 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-08-23 10:53:18,952 DEBUG [RS:0;jenkins-hbase4:37675] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,951 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,953 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,952 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:2;jenkins-hbase4:43499] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,954 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,955 DEBUG [RS:1;jenkins-hbase4:41821] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-08-23 10:53:18,955 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-23 10:53:18,957 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,957 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,957 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,961 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,962 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,962 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,967 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,967 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,967 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,984 INFO [RS:0;jenkins-hbase4:37675] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-23 10:53:18,984 INFO [RS:2;jenkins-hbase4:43499] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-23 10:53:18,984 INFO [RS:1;jenkins-hbase4:41821] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-08-23 10:53:18,988 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37675,1692787997438-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,988 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41821,1692787997489-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:18,988 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43499,1692787997532-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,013 INFO [RS:2;jenkins-hbase4:43499] regionserver.Replication(203): jenkins-hbase4.apache.org,43499,1692787997532 started 2023-08-23 10:53:19,013 INFO [RS:0;jenkins-hbase4:37675] regionserver.Replication(203): jenkins-hbase4.apache.org,37675,1692787997438 started 2023-08-23 10:53:19,013 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,43499,1692787997532, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43499, sessionid=0x10038c62a350003 2023-08-23 10:53:19,013 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37675,1692787997438, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37675, sessionid=0x10038c62a350001 2023-08-23 10:53:19,013 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-23 10:53:19,013 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-23 10:53:19,014 DEBUG [RS:2;jenkins-hbase4:43499] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:19,014 DEBUG [RS:0;jenkins-hbase4:37675] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:19,014 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43499,1692787997532' 2023-08-23 10:53:19,015 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37675,1692787997438' 2023-08-23 10:53:19,015 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:19,015 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:19,016 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:19,016 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:19,017 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-23 10:53:19,017 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-23 10:53:19,017 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-23 10:53:19,017 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-23 10:53:19,017 DEBUG [RS:0;jenkins-hbase4:37675] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:19,017 DEBUG [RS:2;jenkins-hbase4:43499] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:19,018 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43499,1692787997532' 2023-08-23 10:53:19,018 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-23 10:53:19,018 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37675,1692787997438' 2023-08-23 10:53:19,018 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-23 10:53:19,018 DEBUG [RS:2;jenkins-hbase4:43499] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-23 10:53:19,019 DEBUG [RS:0;jenkins-hbase4:37675] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-23 10:53:19,019 DEBUG [RS:2;jenkins-hbase4:43499] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-23 10:53:19,019 INFO [RS:2;jenkins-hbase4:43499] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-23 10:53:19,019 INFO [RS:2;jenkins-hbase4:43499] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-23 10:53:19,019 DEBUG [RS:0;jenkins-hbase4:37675] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-23 10:53:19,019 INFO [RS:0;jenkins-hbase4:37675] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-23 10:53:19,021 INFO [RS:1;jenkins-hbase4:41821] regionserver.Replication(203): jenkins-hbase4.apache.org,41821,1692787997489 started 2023-08-23 10:53:19,021 INFO [RS:0;jenkins-hbase4:37675] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-23 10:53:19,022 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41821,1692787997489, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41821, sessionid=0x10038c62a350002 2023-08-23 10:53:19,022 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-08-23 10:53:19,022 DEBUG [RS:1;jenkins-hbase4:41821] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:19,022 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41821,1692787997489' 2023-08-23 10:53:19,022 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:19,023 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41821,1692787997489' 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-08-23 10:53:19,024 DEBUG [RS:1;jenkins-hbase4:41821] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-08-23 10:53:19,025 DEBUG [RS:1;jenkins-hbase4:41821] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-08-23 10:53:19,025 INFO [RS:1;jenkins-hbase4:41821] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-08-23 10:53:19,025 INFO [RS:1;jenkins-hbase4:41821] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-08-23 10:53:19,109 DEBUG [jenkins-hbase4:44301] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-08-23 10:53:19,112 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:19,118 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:19,118 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:19,118 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:19,118 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:19,122 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41821,1692787997489, state=OPENING 2023-08-23 10:53:19,130 INFO [RS:2;jenkins-hbase4:43499] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43499%2C1692787997532, suffix=, logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,43499,1692787997532, archiveDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs, maxLogs=32 2023-08-23 10:53:19,130 INFO [RS:0;jenkins-hbase4:37675] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37675%2C1692787997438, suffix=, logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,37675,1692787997438, archiveDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs, maxLogs=32 2023-08-23 10:53:19,132 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-08-23 10:53:19,132 INFO [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41821%2C1692787997489, suffix=, logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,41821,1692787997489, archiveDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs, maxLogs=32 2023-08-23 10:53:19,134 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:19,135 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-23 10:53:19,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:19,157 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK] 2023-08-23 10:53:19,158 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK] 2023-08-23 10:53:19,160 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK] 2023-08-23 10:53:19,184 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK] 2023-08-23 10:53:19,184 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK] 2023-08-23 10:53:19,185 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK] 2023-08-23 10:53:19,194 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK] 2023-08-23 10:53:19,194 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK] 2023-08-23 10:53:19,195 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK] 2023-08-23 10:53:19,196 INFO [RS:2;jenkins-hbase4:43499] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,43499,1692787997532/jenkins-hbase4.apache.org%2C43499%2C1692787997532.1692787999138 2023-08-23 10:53:19,213 DEBUG [RS:2;jenkins-hbase4:43499] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK], DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK], DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK]] 2023-08-23 10:53:19,220 INFO [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,41821,1692787997489/jenkins-hbase4.apache.org%2C41821%2C1692787997489.1692787999137 2023-08-23 10:53:19,221 INFO [RS:0;jenkins-hbase4:37675] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,37675,1692787997438/jenkins-hbase4.apache.org%2C37675%2C1692787997438.1692787999137 2023-08-23 10:53:19,221 DEBUG [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK], DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK], DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK]] 2023-08-23 10:53:19,225 DEBUG [RS:0;jenkins-hbase4:37675] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK], DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK], DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK]] 2023-08-23 10:53:19,335 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:19,338 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-23 10:53:19,342 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40280, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-23 10:53:19,358 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-08-23 10:53:19,358 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-08-23 10:53:19,362 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41821%2C1692787997489.meta, suffix=.meta, logDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,41821,1692787997489, archiveDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs, maxLogs=32 2023-08-23 10:53:19,384 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK] 2023-08-23 10:53:19,385 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK] 2023-08-23 10:53:19,387 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK] 2023-08-23 10:53:19,392 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/WALs/jenkins-hbase4.apache.org,41821,1692787997489/jenkins-hbase4.apache.org%2C41821%2C1692787997489.meta.1692787999364.meta 2023-08-23 10:53:19,393 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41707,DS-dc374cc7-e08a-46e7-affa-82501381bb96,DISK], DatanodeInfoWithStorage[127.0.0.1:44203,DS-ecd5478c-a904-408a-a1d1-c5e8b8fe9775,DISK], DatanodeInfoWithStorage[127.0.0.1:45527,DS-718d104c-374b-40e2-b5ff-b6546bb4eddf,DISK]] 2023-08-23 10:53:19,393 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:19,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:19,400 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-08-23 10:53:19,419 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-08-23 10:53:19,421 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-08-23 10:53:19,426 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-08-23 10:53:19,426 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:19,426 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-08-23 10:53:19,426 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-08-23 10:53:19,430 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-08-23 10:53:19,432 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/info 2023-08-23 10:53:19,432 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/info 2023-08-23 10:53:19,433 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-08-23 10:53:19,434 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:19,434 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-08-23 10:53:19,435 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/rep_barrier 2023-08-23 10:53:19,435 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/rep_barrier 2023-08-23 10:53:19,436 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-08-23 10:53:19,437 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:19,437 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-08-23 10:53:19,438 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/table 2023-08-23 10:53:19,438 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/table 2023-08-23 10:53:19,439 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-08-23 10:53:19,440 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:19,442 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740 2023-08-23 10:53:19,446 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740 2023-08-23 10:53:19,451 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-08-23 10:53:19,455 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-08-23 10:53:19,457 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67831189, jitterRate=0.010763481259346008}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-08-23 10:53:19,457 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-08-23 10:53:19,477 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1692787999328 2023-08-23 10:53:19,495 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-08-23 10:53:19,495 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-08-23 10:53:19,496 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41821,1692787997489, state=OPEN 2023-08-23 10:53:19,499 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-08-23 10:53:19,499 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-08-23 10:53:19,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-08-23 10:53:19,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41821,1692787997489 in 360 msec 2023-08-23 10:53:19,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-08-23 10:53:19,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 578 msec 2023-08-23 10:53:19,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 893 msec 2023-08-23 10:53:19,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1692787999515, completionTime=-1 2023-08-23 10:53:19,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-08-23 10:53:19,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:19,581 DEBUG [hconnection-0x7797637d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-23 10:53:19,584 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40284, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-23 10:53:19,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:19,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788059606 2023-08-23 10:53:19,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788119606 2023-08-23 10:53:19,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 91 msec 2023-08-23 10:53:19,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44301,1692787996219-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44301,1692787996219-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44301,1692787996219-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44301, period=300000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-08-23 10:53:19,643 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-08-23 10:53:19,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-08-23 10:53:19,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-08-23 10:53:19,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:19,675 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:19,678 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:19,701 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/hbase/namespace/03da722d073e4770f60de66574c27a90 2023-08-23 10:53:19,703 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/hbase/namespace/03da722d073e4770f60de66574c27a90 empty. 2023-08-23 10:53:19,704 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/hbase/namespace/03da722d073e4770f60de66574c27a90 2023-08-23 10:53:19,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-08-23 10:53:19,744 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:19,746 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 03da722d073e4770f60de66574c27a90, NAME => 'hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 03da722d073e4770f60de66574c27a90, disabling compactions & flushes 2023-08-23 10:53:19,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. after waiting 0 ms 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:19,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:19,770 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 03da722d073e4770f60de66574c27a90: 2023-08-23 10:53:19,775 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:19,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692787999777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692787999777"}]},"ts":"1692787999777"} 2023-08-23 10:53:19,818 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:19,820 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:19,824 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692787999820"}]},"ts":"1692787999820"} 2023-08-23 10:53:19,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-08-23 10:53:19,835 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:19,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:19,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:19,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:19,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:19,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=03da722d073e4770f60de66574c27a90, ASSIGN}] 2023-08-23 10:53:19,848 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=03da722d073e4770f60de66574c27a90, ASSIGN 2023-08-23 10:53:19,850 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=03da722d073e4770f60de66574c27a90, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37675,1692787997438; forceNewPlan=false, retain=false 2023-08-23 10:53:20,003 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:20,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=03da722d073e4770f60de66574c27a90, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:20,004 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692788000004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788000004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788000004"}]},"ts":"1692788000004"} 2023-08-23 10:53:20,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 03da722d073e4770f60de66574c27a90, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:20,164 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:20,164 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-23 10:53:20,169 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36210, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-23 10:53:20,177 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:20,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 03da722d073e4770f60de66574c27a90, NAME => 'hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:20,178 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:20,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:20,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,182 INFO [StoreOpener-03da722d073e4770f60de66574c27a90-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,184 DEBUG [StoreOpener-03da722d073e4770f60de66574c27a90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/info 2023-08-23 10:53:20,184 DEBUG [StoreOpener-03da722d073e4770f60de66574c27a90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/info 2023-08-23 10:53:20,184 INFO [StoreOpener-03da722d073e4770f60de66574c27a90-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03da722d073e4770f60de66574c27a90 columnFamilyName info 2023-08-23 10:53:20,185 INFO [StoreOpener-03da722d073e4770f60de66574c27a90-1] regionserver.HStore(310): Store=03da722d073e4770f60de66574c27a90/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:20,187 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,188 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,192 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 03da722d073e4770f60de66574c27a90 2023-08-23 10:53:20,196 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:20,197 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 03da722d073e4770f60de66574c27a90; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61559349, jitterRate=-0.08269421756267548}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-23 10:53:20,197 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 03da722d073e4770f60de66574c27a90: 2023-08-23 10:53:20,200 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90., pid=6, masterSystemTime=1692788000164 2023-08-23 10:53:20,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:20,205 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:53:20,207 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=03da722d073e4770f60de66574c27a90, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:20,207 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1692788000206"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788000206"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788000206"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788000206"}]},"ts":"1692788000206"} 2023-08-23 10:53:20,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-08-23 10:53:20,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 03da722d073e4770f60de66574c27a90, server=jenkins-hbase4.apache.org,37675,1692787997438 in 202 msec 2023-08-23 10:53:20,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-08-23 10:53:20,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=03da722d073e4770f60de66574c27a90, ASSIGN in 375 msec 2023-08-23 10:53:20,220 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:20,220 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788000220"}]},"ts":"1692788000220"} 2023-08-23 10:53:20,222 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-08-23 10:53:20,226 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:20,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 565 msec 2023-08-23 10:53:20,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-08-23 10:53:20,277 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-08-23 10:53:20,277 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:20,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-23 10:53:20,302 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-23 10:53:20,320 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:20,338 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-23 10:53:20,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 35 msec 2023-08-23 10:53:20,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:20,364 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-08-23 10:53:20,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-08-23 10:53:20,380 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-08-23 10:53:20,382 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-08-23 10:53:20,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.774sec 2023-08-23 10:53:20,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-08-23 10:53:20,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-08-23 10:53:20,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-08-23 10:53:20,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44301,1692787996219-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-08-23 10:53:20,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44301,1692787996219-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-08-23 10:53:20,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-08-23 10:53:20,474 DEBUG [Listener at localhost/34585] zookeeper.ReadOnlyZKClient(139): Connect 0x53986fab to 127.0.0.1:52557 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-08-23 10:53:20,481 DEBUG [Listener at localhost/34585] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f8d6a62, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-08-23 10:53:20,501 DEBUG [hconnection-0x6c60eea6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-23 10:53:20,512 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40294, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-23 10:53:20,525 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:53:20,562 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=364, OpenFileDescriptor=698, MaxFileDescriptor=60000, SystemLoadAverage=322, ProcessCount=171, AvailableMemoryMB=11055 2023-08-23 10:53:20,566 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:20,576 DEBUG [Listener at localhost/34585] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-08-23 10:53:20,579 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47618, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-08-23 10:53:20,594 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:20,594 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:53:20,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:20,602 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:20,604 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:20,606 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:20,609 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:20,610 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b empty. 2023-08-23 10:53:20,612 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:20,612 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-23 10:53:21,035 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:21,037 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 83b16afbbfa1198d46801c8c70153e1b, NAME => 'testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 83b16afbbfa1198d46801c8c70153e1b, disabling compactions & flushes 2023-08-23 10:53:21,062 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. after waiting 0 ms 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,062 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,062 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:21,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:21,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788001068"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788001068"}]},"ts":"1692788001068"} 2023-08-23 10:53:21,070 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:21,072 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:21,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788001072"}]},"ts":"1692788001072"} 2023-08-23 10:53:21,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-08-23 10:53:21,079 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:21,081 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:21,081 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:21,081 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:21,081 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:21,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, ASSIGN}] 2023-08-23 10:53:21,084 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, ASSIGN 2023-08-23 10:53:21,085 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:21,236 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:21,237 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=83b16afbbfa1198d46801c8c70153e1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788001237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788001237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788001237"}]},"ts":"1692788001237"} 2023-08-23 10:53:21,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 83b16afbbfa1198d46801c8c70153e1b, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:21,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 83b16afbbfa1198d46801c8c70153e1b, NAME => 'testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:21,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,403 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,405 DEBUG [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1 2023-08-23 10:53:21,406 DEBUG [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1 2023-08-23 10:53:21,406 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83b16afbbfa1198d46801c8c70153e1b columnFamilyName cf1 2023-08-23 10:53:21,407 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] regionserver.HStore(310): Store=83b16afbbfa1198d46801c8c70153e1b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:21,407 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,409 DEBUG [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2 2023-08-23 10:53:21,409 DEBUG [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2 2023-08-23 10:53:21,410 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83b16afbbfa1198d46801c8c70153e1b columnFamilyName cf2 2023-08-23 10:53:21,410 INFO [StoreOpener-83b16afbbfa1198d46801c8c70153e1b-1] regionserver.HStore(310): Store=83b16afbbfa1198d46801c8c70153e1b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:21,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:21,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:21,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:21,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 83b16afbbfa1198d46801c8c70153e1b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75022539, jitterRate=0.11792294681072235}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:21,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:21,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b., pid=11, masterSystemTime=1692788001393 2023-08-23 10:53:21,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=83b16afbbfa1198d46801c8c70153e1b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,430 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788001430"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788001430"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788001430"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788001430"}]},"ts":"1692788001430"} 2023-08-23 10:53:21,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-08-23 10:53:21,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 83b16afbbfa1198d46801c8c70153e1b, server=jenkins-hbase4.apache.org,41821,1692787997489 in 193 msec 2023-08-23 10:53:21,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-08-23 10:53:21,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, ASSIGN in 356 msec 2023-08-23 10:53:21,442 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:21,442 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788001442"}]},"ts":"1692788001442"} 2023-08-23 10:53:21,445 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-08-23 10:53:21,448 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:21,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 848 msec 2023-08-23 10:53:21,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:21,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,639 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:21,639 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:21,640 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:21,640 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:21,640 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,641 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,642 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,643 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,643 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:21,647 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:21,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:21,648 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:21,650 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,650 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:21,651 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,652 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:53:21,653 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,653 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,653 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:21,653 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,653 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:21,728 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/fcdb3a6196eb457385a0d826e8b141c0 2023-08-23 10:53:21,791 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/5833a653864a463fa1ec59739d4f97e1 2023-08-23 10:53:21,802 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/fcdb3a6196eb457385a0d826e8b141c0 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0 2023-08-23 10:53:21,812 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:21,815 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/5833a653864a463fa1ec59739d4f97e1 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1 2023-08-23 10:53:21,824 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:21,826 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 174ms, sequenceid=6, compaction requested=false 2023-08-23 10:53:21,828 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:21,829 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:21,830 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:21,830 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:21,830 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:21,830 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,830 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:21,830 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:21,832 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,832 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:21,833 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,833 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:21,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:21,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:21,836 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:21,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43616160[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:21,836 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,840 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,840 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,840 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:21,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,841 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:21,841 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:21,846 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,846 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:21,846 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:21,846 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:21,847 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:21,847 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:21,847 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:21,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:21,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:21,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:21,854 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:21,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:21,854 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:21,854 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:21,854 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,854 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:21,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:21,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:21,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,858 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:21,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:21,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,863 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,863 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,863 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,863 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,864 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:21,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,876 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:21,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:21,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:21,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:21,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:21,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:21,878 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,878 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:21,878 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:21,878 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,878 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:21,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,880 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:21,880 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:21,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:21,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:21,980 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:21,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:22,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:22,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:22,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,016 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,017 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,018 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:22,018 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:22,018 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,018 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,021 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,021 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,021 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:22,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,023 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,023 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,025 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,026 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,027 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,027 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,027 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,028 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,028 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,028 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,029 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:22,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,030 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,031 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,031 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,048 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/e083541f5e9042818732f4c23df858ad 2023-08-23 10:53:22,074 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/ba6e55d416074b0e8ba8100f6229fb2c 2023-08-23 10:53:22,083 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/e083541f5e9042818732f4c23df858ad as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad 2023-08-23 10:53:22,093 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:22,094 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/ba6e55d416074b0e8ba8100f6229fb2c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c 2023-08-23 10:53:22,105 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:22,106 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 77ms, sequenceid=19, compaction requested=false 2023-08-23 10:53:22,106 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:22,106 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,106 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:22,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:22,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,109 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,109 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,110 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,110 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:22,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38c93f2b[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:22,112 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:22,112 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:22,115 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:22,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,116 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,116 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,116 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:22,118 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,118 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,118 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,118 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:22,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:22,124 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:22,124 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:22,124 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:22,124 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,124 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,128 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,130 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,131 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,135 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,136 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,137 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:22,148 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,148 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,149 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,150 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:22,150 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,149 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,150 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,151 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,250 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:22,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:22,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:22,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:22,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,277 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,277 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,278 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:22,278 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:22,279 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,279 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,280 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,281 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,281 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,281 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,281 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:22,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,283 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,283 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,285 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,286 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:22,287 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,287 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:22,289 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,290 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,290 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,290 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,290 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,290 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,312 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/dde1c7b9e75041ae916f4a84e830341e 2023-08-23 10:53:22,338 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/10b9d80a5482401a9cefe836a1a4a784 2023-08-23 10:53:22,348 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/dde1c7b9e75041ae916f4a84e830341e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e 2023-08-23 10:53:22,356 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:22,357 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/10b9d80a5482401a9cefe836a1a4a784 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784 2023-08-23 10:53:22,365 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:22,366 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 79ms, sequenceid=32, compaction requested=true 2023-08-23 10:53:22,367 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:22,367 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,367 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:22,368 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:22,368 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,368 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,368 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,371 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,371 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,371 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,371 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:22,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b291eca[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:22,374 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:22,374 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,376 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,376 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:22,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:22,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,376 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:22,379 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,379 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,379 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,380 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,380 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,380 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,380 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,380 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,380 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:22,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:22,386 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:22,386 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:22,386 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:22,386 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,386 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:22,389 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,390 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,389 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,391 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,391 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,393 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,394 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,394 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,400 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,400 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,400 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:22,400 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,400 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,401 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,401 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,401 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,401 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,401 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:22,402 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:22,502 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:22,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:22,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:22,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:22,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,530 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,530 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,531 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:22,531 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:22,531 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,531 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,533 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,533 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,534 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,535 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,535 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,536 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,535 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,536 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,536 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,537 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,537 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:22,537 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,537 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:22,537 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:22,537 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,538 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,537 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,538 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,538 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,538 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:22,540 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,541 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,542 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,542 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,542 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,542 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,562 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/cbc8c281906c432693c62d4bc194c673 2023-08-23 10:53:22,593 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/bf27c40424da4f4e968c2702b9ef3c0e 2023-08-23 10:53:22,602 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/cbc8c281906c432693c62d4bc194c673 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673 2023-08-23 10:53:22,611 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:22,612 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/bf27c40424da4f4e968c2702b9ef3c0e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e 2023-08-23 10:53:22,620 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:22,622 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 84ms, sequenceid=45, compaction requested=true 2023-08-23 10:53:22,622 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:22,622 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,622 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:22,623 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:22,623 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,623 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,623 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,631 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,631 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,631 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,631 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:22,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f75e88c[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:22,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:22,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,635 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,636 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,636 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:22,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:22,637 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,637 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,637 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,639 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:22,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:22,646 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:22,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:22,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:22,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,646 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,649 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,648 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,649 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,650 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,652 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,653 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,653 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,659 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:22,657 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,659 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:22,658 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,658 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,659 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,659 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,659 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,759 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:22,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:22,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:22,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:22,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,784 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,784 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,785 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:22,785 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:22,786 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,786 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,788 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,788 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,789 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,789 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,791 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,791 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,791 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,791 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,792 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,792 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:22,792 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:22,794 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,794 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,794 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:22,796 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,796 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,797 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,797 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,797 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,797 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,814 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/3611525174bf440b86e4844948ef90ee 2023-08-23 10:53:22,842 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/95803de80ee74fbf85aaa448e6dd1425 2023-08-23 10:53:22,851 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/3611525174bf440b86e4844948ef90ee as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee 2023-08-23 10:53:22,859 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:22,860 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/95803de80ee74fbf85aaa448e6dd1425 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425 2023-08-23 10:53:22,868 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:22,869 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 75ms, sequenceid=58, compaction requested=true 2023-08-23 10:53:22,869 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:22,869 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:22,869 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,871 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:22,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:22,873 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,874 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,874 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:22,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:22,877 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:22,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4757e263[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:22,877 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,879 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,879 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:22,879 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:22,882 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,882 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,882 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:22,883 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,883 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:22,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:22,888 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:22,888 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:22,888 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:22,888 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,888 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:22,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,890 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,891 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,891 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,891 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,892 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:22,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:22,894 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,895 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,895 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,902 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:22,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,903 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:22,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:22,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:22,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:22,903 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,904 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:22,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:22,904 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:22,905 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:22,906 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,005 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:23,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:23,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:23,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:23,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,030 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:23,030 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:23,032 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-08-23 10:53:23,032 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-08-23 10:53:23,032 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,032 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,033 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,034 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,033 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,033 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,034 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,034 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:23,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:23,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:23,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:23,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:23,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:23,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:23,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-08-23 10:53:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:23,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-08-23 10:53:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:23,039 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-08-23 10:53:23,039 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:23,039 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:23,039 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,039 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. started... 2023-08-23 10:53:23,039 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:23,040 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,043 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,043 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:23,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:23,062 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/576f25125f2f4a1b960f99ee2bee922b 2023-08-23 10:53:23,087 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/094b2552210e44859a2be722d4dca169 2023-08-23 10:53:23,095 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/576f25125f2f4a1b960f99ee2bee922b as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b 2023-08-23 10:53:23,103 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:23,104 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/094b2552210e44859a2be722d4dca169 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169 2023-08-23 10:53:23,112 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:23,113 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 74ms, sequenceid=71, compaction requested=true 2023-08-23 10:53:23,113 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:23,113 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,113 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-08-23 10:53:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-08-23 10:53:23,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:23,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-08-23 10:53:23,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3696bdc5[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:23,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:23,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,120 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:23,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:23,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:23,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-08-23 10:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:23,122 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:23,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:23,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:23,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:23,125 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-08-23 10:53:23,125 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:23,125 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-08-23 10:53:23,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:23,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:23,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:23,129 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-08-23 10:53:23,129 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:23,129 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:23,129 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,129 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:23,132 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:23,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,133 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,133 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,135 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:23,136 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,136 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:23,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:23,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:23,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-08-23 10:53:23,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:23,139 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,139 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:23,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,140 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:23,141 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,142 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-08-23 10:53:23,241 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:23,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:23,265 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,271 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:23,272 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:23,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:23,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:23,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, UNASSIGN}] 2023-08-23 10:53:23,281 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, UNASSIGN 2023-08-23 10:53:23,282 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=83b16afbbfa1198d46801c8c70153e1b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:23,282 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788003282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788003282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788003282"}]},"ts":"1692788003282"} 2023-08-23 10:53:23,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 83b16afbbfa1198d46801c8c70153e1b, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:23,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 83b16afbbfa1198d46801c8c70153e1b, disabling compactions & flushes 2023-08-23 10:53:23,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. after waiting 0 ms 2023-08-23 10:53:23,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 83b16afbbfa1198d46801c8c70153e1b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:53:23,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 2023-08-23 10:53:23,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/bc7c133f218440738922527446310127 2023-08-23 10:53:23,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 2023-08-23 10:53:23,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:23,506 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.tmp/cf2/bc7c133f218440738922527446310127 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127 2023-08-23 10:53:23,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:23,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 83b16afbbfa1198d46801c8c70153e1b in 71ms, sequenceid=82, compaction requested=true 2023-08-23 10:53:23,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:53:23,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:23,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. 2023-08-23 10:53:23,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 83b16afbbfa1198d46801c8c70153e1b: 2023-08-23 10:53:23,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=83b16afbbfa1198d46801c8c70153e1b, regionState=CLOSED 2023-08-23 10:53:23,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788003547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788003547"}]},"ts":"1692788003547"} 2023-08-23 10:53:23,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-08-23 10:53:23,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 83b16afbbfa1198d46801c8c70153e1b, server=jenkins-hbase4.apache.org,41821,1692787997489 in 265 msec 2023-08-23 10:53:23,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-08-23 10:53:23,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=83b16afbbfa1198d46801c8c70153e1b, UNASSIGN in 273 msec 2023-08-23 10:53:23,576 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=83b16afbbfa1198d46801c8c70153e1b, threads=14 2023-08-23 10:53:23,577 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,577 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,577 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,577 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,578 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,578 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,578 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,578 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,581 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,581 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,585 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,585 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,585 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,585 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,608 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181, top=true 2023-08-23 10:53:23,626 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169, top=true 2023-08-23 10:53:23,654 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425, top=true 2023-08-23 10:53:23,678 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127, top=true 2023-08-23 10:53:23,678 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee, top=true 2023-08-23 10:53:23,691 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181 for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,691 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,702 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169 for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,702 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,705 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b, top=true 2023-08-23 10:53:23,727 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425 for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,727 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,742 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127 for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,742 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,745 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,745 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,761 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/.splits/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee for child: 196b1478f733e5db2758765b9cd2bb43, parent: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,762 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,845 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,849 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,866 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,867 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,884 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0 for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,884 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,884 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,887 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad for region: 83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:23,887 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 83b16afbbfa1198d46801c8c70153e1b Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:23,945 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:23,948 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:23,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1692788003951"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1692788003951"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1692788003951"}]},"ts":"1692788003951"} 2023-08-23 10:53:23,952 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788003951"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788003951"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788003951"}]},"ts":"1692788003951"} 2023-08-23 10:53:23,952 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788003951"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788003951"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788003951"}]},"ts":"1692788003951"} 2023-08-23 10:53:24,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, ASSIGN}] 2023-08-23 10:53:24,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, ASSIGN 2023-08-23 10:53:24,005 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:24,006 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, ASSIGN 2023-08-23 10:53:24,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:24,155 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:53:24,157 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f1547d5000cdc5a6e9fd38083a69821a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:24,158 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788004157"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788004157"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788004157"}]},"ts":"1692788004157"} 2023-08-23 10:53:24,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure f1547d5000cdc5a6e9fd38083a69821a, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:24,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=196b1478f733e5db2758765b9cd2bb43, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:24,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788004165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788004165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788004165"}]},"ts":"1692788004165"} 2023-08-23 10:53:24,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 196b1478f733e5db2758765b9cd2bb43, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:24,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 196b1478f733e5db2758765b9cd2bb43, NAME => 'testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.', STARTKEY => '26', ENDKEY => ''} 2023-08-23 10:53:24,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:24,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:24,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,321 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,322 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1 2023-08-23 10:53:24,322 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1 2023-08-23 10:53:24,323 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 196b1478f733e5db2758765b9cd2bb43 columnFamilyName cf1 2023-08-23 10:53:24,361 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-top 2023-08-23 10:53:24,368 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-top 2023-08-23 10:53:24,375 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-top 2023-08-23 10:53:24,381 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-top 2023-08-23 10:53:24,388 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181 2023-08-23 10:53:24,393 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee 2023-08-23 10:53:24,398 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b 2023-08-23 10:53:24,398 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(310): Store=196b1478f733e5db2758765b9cd2bb43/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:24,399 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,400 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2 2023-08-23 10:53:24,400 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2 2023-08-23 10:53:24,400 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 196b1478f733e5db2758765b9cd2bb43 columnFamilyName cf2 2023-08-23 10:53:24,414 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-top 2023-08-23 10:53:24,419 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-top 2023-08-23 10:53:24,425 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-top 2023-08-23 10:53:24,430 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-top 2023-08-23 10:53:24,437 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169 2023-08-23 10:53:24,452 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425 2023-08-23 10:53:24,465 DEBUG [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127 2023-08-23 10:53:24,466 INFO [StoreOpener-196b1478f733e5db2758765b9cd2bb43-1] regionserver.HStore(310): Store=196b1478f733e5db2758765b9cd2bb43/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:24,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:24,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:24,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 196b1478f733e5db2758765b9cd2bb43; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73626470, jitterRate=0.09711989760398865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:24,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 196b1478f733e5db2758765b9cd2bb43: 2023-08-23 10:53:24,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43., pid=18, masterSystemTime=1692788004313 2023-08-23 10:53:24,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:24,480 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:24,480 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:24,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,482 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,482 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1912): 196b1478f733e5db2758765b9cd2bb43/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:24,482 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,483 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 196b1478f733e5db2758765b9cd2bb43/cf1 in testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,483 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1912): 196b1478f733e5db2758765b9cd2bb43/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:24,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1547d5000cdc5a6e9fd38083a69821a, NAME => 'testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.', STARTKEY => '', ENDKEY => '26'} 2023-08-23 10:53:24,483 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 196b1478f733e5db2758765b9cd2bb43/cf2 in testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:24,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:24,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,483 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/.tmp, totalSize=34.9 K 2023-08-23 10:53:24,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:24,483 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/.tmp, totalSize=34.9 K 2023-08-23 10:53:24,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,485 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=196b1478f733e5db2758765b9cd2bb43, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:24,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788004485"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788004485"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788004485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788004485"}]},"ts":"1692788004485"} 2023-08-23 10:53:24,487 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788001608 2023-08-23 10:53:24,488 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788001608 2023-08-23 10:53:24,488 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788001989 2023-08-23 10:53:24,488 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,490 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788001989 2023-08-23 10:53:24,491 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788002253 2023-08-23 10:53:24,492 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788002253 2023-08-23 10:53:24,492 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788002505 2023-08-23 10:53:24,493 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788002505 2023-08-23 10:53:24,493 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788002762 2023-08-23 10:53:24,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-08-23 10:53:24,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 196b1478f733e5db2758765b9cd2bb43, server=jenkins-hbase4.apache.org,41821,1692787997489 in 320 msec 2023-08-23 10:53:24,494 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788002762 2023-08-23 10:53:24,495 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788003008 2023-08-23 10:53:24,496 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788003244 2023-08-23 10:53:24,496 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788003008 2023-08-23 10:53:24,496 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788003244 2023-08-23 10:53:24,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, ASSIGN in 491 msec 2023-08-23 10:53:24,503 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1 2023-08-23 10:53:24,503 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1 2023-08-23 10:53:24,503 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1547d5000cdc5a6e9fd38083a69821a columnFamilyName cf1 2023-08-23 10:53:24,529 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-bottom 2023-08-23 10:53:24,543 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-bottom 2023-08-23 10:53:24,548 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): 196b1478f733e5db2758765b9cd2bb43#cf1#compaction#14 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:24,555 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): 196b1478f733e5db2758765b9cd2bb43#cf2#compaction#15 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:24,556 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-bottom 2023-08-23 10:53:24,561 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-bottom 2023-08-23 10:53:24,561 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(310): Store=f1547d5000cdc5a6e9fd38083a69821a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:24,562 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,563 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2 2023-08-23 10:53:24,563 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2 2023-08-23 10:53:24,564 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1547d5000cdc5a6e9fd38083a69821a columnFamilyName cf2 2023-08-23 10:53:24,587 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-bottom 2023-08-23 10:53:24,591 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/.tmp/cf2/54970144bd1e499db7af5c37ce3c96dd as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/54970144bd1e499db7af5c37ce3c96dd 2023-08-23 10:53:24,597 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-bottom 2023-08-23 10:53:24,601 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/.tmp/cf1/400c102a42c748eaa98c13887759d7ac as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/400c102a42c748eaa98c13887759d7ac 2023-08-23 10:53:24,605 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-bottom 2023-08-23 10:53:24,613 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 196b1478f733e5db2758765b9cd2bb43/cf2 of 196b1478f733e5db2758765b9cd2bb43 into 54970144bd1e499db7af5c37ce3c96dd(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:24,613 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 196b1478f733e5db2758765b9cd2bb43/cf1 of 196b1478f733e5db2758765b9cd2bb43 into 400c102a42c748eaa98c13887759d7ac(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:24,614 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 196b1478f733e5db2758765b9cd2bb43: 2023-08-23 10:53:24,614 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 196b1478f733e5db2758765b9cd2bb43: 2023-08-23 10:53:24,614 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43., storeName=196b1478f733e5db2758765b9cd2bb43/cf2, priority=9, startTime=1692788004479; duration=0sec 2023-08-23 10:53:24,614 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43., storeName=196b1478f733e5db2758765b9cd2bb43/cf1, priority=9, startTime=1692788004479; duration=0sec 2023-08-23 10:53:24,615 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,615 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,617 DEBUG [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-bottom 2023-08-23 10:53:24,617 INFO [StoreOpener-f1547d5000cdc5a6e9fd38083a69821a-1] regionserver.HStore(310): Store=f1547d5000cdc5a6e9fd38083a69821a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:24,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:24,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:24,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f1547d5000cdc5a6e9fd38083a69821a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69396281, jitterRate=0.034085169434547424}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:24,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f1547d5000cdc5a6e9fd38083a69821a: 2023-08-23 10:53:24,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a., pid=17, masterSystemTime=1692788004313 2023-08-23 10:53:24,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,626 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-23 10:53:24,626 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-08-23 10:53:24,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:24,626 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,626 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1912): f1547d5000cdc5a6e9fd38083a69821a/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:24,626 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,626 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f1547d5000cdc5a6e9fd38083a69821a/cf1 in testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,626 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1912): f1547d5000cdc5a6e9fd38083a69821a/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:24,627 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f1547d5000cdc5a6e9fd38083a69821a/cf2 in testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,627 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/.tmp, totalSize=19.9 K 2023-08-23 10:53:24,627 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/.tmp, totalSize=19.9 K 2023-08-23 10:53:24,628 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788001608 2023-08-23 10:53:24,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:24,630 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788001608 2023-08-23 10:53:24,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=f1547d5000cdc5a6e9fd38083a69821a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:24,631 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788001989 2023-08-23 10:53:24,631 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788001989 2023-08-23 10:53:24,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788004631"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788004631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788004631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788004631"}]},"ts":"1692788004631"} 2023-08-23 10:53:24,632 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788002253 2023-08-23 10:53:24,633 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788002253 2023-08-23 10:53:24,633 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788002505 2023-08-23 10:53:24,635 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788002505 2023-08-23 10:53:24,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-08-23 10:53:24,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure f1547d5000cdc5a6e9fd38083a69821a, server=jenkins-hbase4.apache.org,41821,1692787997489 in 475 msec 2023-08-23 10:53:24,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-08-23 10:53:24,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, ASSIGN in 636 msec 2023-08-23 10:53:24,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 in 1.3770 sec 2023-08-23 10:53:24,654 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): f1547d5000cdc5a6e9fd38083a69821a#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:24,654 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): f1547d5000cdc5a6e9fd38083a69821a#cf2#compaction#17 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:24,676 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/.tmp/cf2/8a39b6a5993b494795a0c05841d64ebc as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/8a39b6a5993b494795a0c05841d64ebc 2023-08-23 10:53:24,679 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/.tmp/cf1/2d3f3b79994e4badb7cf7bc0eba1f964 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/2d3f3b79994e4badb7cf7bc0eba1f964 2023-08-23 10:53:24,684 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f1547d5000cdc5a6e9fd38083a69821a/cf2 of f1547d5000cdc5a6e9fd38083a69821a into 8a39b6a5993b494795a0c05841d64ebc(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:24,686 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f1547d5000cdc5a6e9fd38083a69821a: 2023-08-23 10:53:24,686 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a., storeName=f1547d5000cdc5a6e9fd38083a69821a/cf2, priority=12, startTime=1692788004626; duration=0sec 2023-08-23 10:53:24,686 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,690 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f1547d5000cdc5a6e9fd38083a69821a/cf1 of f1547d5000cdc5a6e9fd38083a69821a into 2d3f3b79994e4badb7cf7bc0eba1f964(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:24,690 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f1547d5000cdc5a6e9fd38083a69821a: 2023-08-23 10:53:24,690 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a., storeName=f1547d5000cdc5a6e9fd38083a69821a/cf1, priority=12, startTime=1692788004625; duration=0sec 2023-08-23 10:53:24,690 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:24,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41821] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40294 deadline: 1692788064773, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b. is not online on jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:24,907 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-23 10:53:24,917 INFO [Listener at localhost/34585] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-23 10:53:24,920 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-08-23 10:53:24,953 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:24,963 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:53:24,966 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-08-23 10:53:24,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-08-23 10:53:24,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:24,995 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788004995"}]},"ts":"1692788004995"} 2023-08-23 10:53:24,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-08-23 10:53:25,005 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-08-23 10:53:25,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-23 10:53:25,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, UNASSIGN}] 2023-08-23 10:53:25,011 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, UNASSIGN 2023-08-23 10:53:25,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=196b1478f733e5db2758765b9cd2bb43, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:25,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788005013"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788005013"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788005013"}]},"ts":"1692788005013"} 2023-08-23 10:53:25,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 196b1478f733e5db2758765b9cd2bb43, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:25,020 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, UNASSIGN 2023-08-23 10:53:25,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f1547d5000cdc5a6e9fd38083a69821a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:25,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788005021"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788005021"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788005021"}]},"ts":"1692788005021"} 2023-08-23 10:53:25,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure f1547d5000cdc5a6e9fd38083a69821a, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:25,063 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-08-23 10:53:25,064 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-08-23 10:53:25,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-23 10:53:25,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:25,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 196b1478f733e5db2758765b9cd2bb43, disabling compactions & flushes 2023-08-23 10:53:25,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:25,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:25,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. after waiting 0 ms 2023-08-23 10:53:25,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:25,181 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181] to archive 2023-08-23 10:53:25,182 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:25,188 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,189 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,191 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,193 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,195 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-3611525174bf440b86e4844948ef90ee 2023-08-23 10:53:25,197 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-576f25125f2f4a1b960f99ee2bee922b 2023-08-23 10:53:25,199 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-1b4c561c8f5d49f1ab7bf5d6613ab181 2023-08-23 10:53:25,218 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127] to archive 2023-08-23 10:53:25,219 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:25,222 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,224 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,226 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,227 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,229 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-95803de80ee74fbf85aaa448e6dd1425 2023-08-23 10:53:25,230 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-094b2552210e44859a2be722d4dca169 2023-08-23 10:53:25,232 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/testSplitTableRegionUnevenDaughter=83b16afbbfa1198d46801c8c70153e1b-bc7c133f218440738922527446310127 2023-08-23 10:53:25,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:53:25,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:25,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43. 2023-08-23 10:53:25,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 196b1478f733e5db2758765b9cd2bb43: 2023-08-23 10:53:25,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:25,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:25,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f1547d5000cdc5a6e9fd38083a69821a, disabling compactions & flushes 2023-08-23 10:53:25,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:25,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:25,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. after waiting 0 ms 2023-08-23 10:53:25,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:25,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=196b1478f733e5db2758765b9cd2bb43, regionState=CLOSED 2023-08-23 10:53:25,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788005242"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788005242"}]},"ts":"1692788005242"} 2023-08-23 10:53:25,243 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673-bottom] to archive 2023-08-23 10:53:25,244 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:25,247 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/fcdb3a6196eb457385a0d826e8b141c0.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-08-23 10:53:25,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 196b1478f733e5db2758765b9cd2bb43, server=jenkins-hbase4.apache.org,41821,1692787997489 in 229 msec 2023-08-23 10:53:25,250 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/e083541f5e9042818732f4c23df858ad.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=196b1478f733e5db2758765b9cd2bb43, UNASSIGN in 239 msec 2023-08-23 10:53:25,251 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/dde1c7b9e75041ae916f4a84e830341e.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,253 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/cbc8c281906c432693c62d4bc194c673.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,255 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e-bottom] to archive 2023-08-23 10:53:25,255 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:25,257 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/5833a653864a463fa1ec59739d4f97e1.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,259 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/ba6e55d416074b0e8ba8100f6229fb2c.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,260 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/10b9d80a5482401a9cefe836a1a4a784.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,262 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/bf27c40424da4f4e968c2702b9ef3c0e.83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:53:25,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:25,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a. 2023-08-23 10:53:25,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f1547d5000cdc5a6e9fd38083a69821a: 2023-08-23 10:53:25,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:25,272 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f1547d5000cdc5a6e9fd38083a69821a, regionState=CLOSED 2023-08-23 10:53:25,272 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1692788005272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788005272"}]},"ts":"1692788005272"} 2023-08-23 10:53:25,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-08-23 10:53:25,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure f1547d5000cdc5a6e9fd38083a69821a, server=jenkins-hbase4.apache.org,41821,1692787997489 in 251 msec 2023-08-23 10:53:25,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-08-23 10:53:25,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1547d5000cdc5a6e9fd38083a69821a, UNASSIGN in 268 msec 2023-08-23 10:53:25,281 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788005281"}]},"ts":"1692788005281"} 2023-08-23 10:53:25,282 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-08-23 10:53:25,284 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-08-23 10:53:25,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 306 msec 2023-08-23 10:53:25,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-08-23 10:53:25,315 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-08-23 10:53:25,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,329 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,331 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-23 10:53:25,340 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,340 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:25,340 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:25,344 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/recovered.edits] 2023-08-23 10:53:25,345 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/recovered.edits] 2023-08-23 10:53:25,345 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/recovered.edits] 2023-08-23 10:53:25,349 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/2d3f3b79994e4badb7cf7bc0eba1f964 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf1/2d3f3b79994e4badb7cf7bc0eba1f964 2023-08-23 10:53:25,350 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/400c102a42c748eaa98c13887759d7ac to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf1/400c102a42c748eaa98c13887759d7ac 2023-08-23 10:53:25,350 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/1b4c561c8f5d49f1ab7bf5d6613ab181 2023-08-23 10:53:25,352 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/3611525174bf440b86e4844948ef90ee 2023-08-23 10:53:25,354 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/8a39b6a5993b494795a0c05841d64ebc to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/cf2/8a39b6a5993b494795a0c05841d64ebc 2023-08-23 10:53:25,356 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/576f25125f2f4a1b960f99ee2bee922b 2023-08-23 10:53:25,359 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/cbc8c281906c432693c62d4bc194c673 2023-08-23 10:53:25,360 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/54970144bd1e499db7af5c37ce3c96dd to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/cf2/54970144bd1e499db7af5c37ce3c96dd 2023-08-23 10:53:25,362 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/dde1c7b9e75041ae916f4a84e830341e 2023-08-23 10:53:25,364 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a/recovered.edits/90.seqid 2023-08-23 10:53:25,364 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/e083541f5e9042818732f4c23df858ad 2023-08-23 10:53:25,365 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1547d5000cdc5a6e9fd38083a69821a 2023-08-23 10:53:25,365 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43/recovered.edits/90.seqid 2023-08-23 10:53:25,366 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:25,366 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf1/fcdb3a6196eb457385a0d826e8b141c0 2023-08-23 10:53:25,369 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/094b2552210e44859a2be722d4dca169 2023-08-23 10:53:25,371 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/10b9d80a5482401a9cefe836a1a4a784 2023-08-23 10:53:25,373 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/5833a653864a463fa1ec59739d4f97e1 2023-08-23 10:53:25,375 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/95803de80ee74fbf85aaa448e6dd1425 2023-08-23 10:53:25,376 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/ba6e55d416074b0e8ba8100f6229fb2c 2023-08-23 10:53:25,378 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bc7c133f218440738922527446310127 2023-08-23 10:53:25,379 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/cf2/bf27c40424da4f4e968c2702b9ef3c0e 2023-08-23 10:53:25,383 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b/recovered.edits/85.seqid 2023-08-23 10:53:25,383 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionUnevenDaughter/83b16afbbfa1198d46801c8c70153e1b 2023-08-23 10:53:25,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-08-23 10:53:25,387 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,416 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-08-23 10:53:25,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-23 10:53:25,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-08-23 10:53:25,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-08-23 10:53:25,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788005462"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:25,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788005462"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:25,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788005462"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:25,466 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:53:25,466 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 83b16afbbfa1198d46801c8c70153e1b, NAME => 'testSplitTableRegionUnevenDaughter,,1692788000600.83b16afbbfa1198d46801c8c70153e1b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f1547d5000cdc5a6e9fd38083a69821a, NAME => 'testSplitTableRegionUnevenDaughter,,1692788003265.f1547d5000cdc5a6e9fd38083a69821a.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 196b1478f733e5db2758765b9cd2bb43, NAME => 'testSplitTableRegionUnevenDaughter,26,1692788003265.196b1478f733e5db2758765b9cd2bb43.', STARTKEY => '26', ENDKEY => ''}] 2023-08-23 10:53:25,466 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-08-23 10:53:25,466 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788005466"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:25,468 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-08-23 10:53:25,470 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:25,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 149 msec 2023-08-23 10:53:25,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-08-23 10:53:25,636 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-08-23 10:53:25,654 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=396 (was 364) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39292 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55858 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39508 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55876 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55556 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55916 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55862 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:41821-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39250 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:54290 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55838 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39200 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:54274 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55844 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:55898 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39192 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39542 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=815 (was 698) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=304 (was 322), ProcessCount=171 (was 171), AvailableMemoryMB=10978 (was 11055) 2023-08-23 10:53:25,671 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=396, OpenFileDescriptor=815, MaxFileDescriptor=60000, SystemLoadAverage=304, ProcessCount=171, AvailableMemoryMB=10977 2023-08-23 10:53:25,671 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:25,682 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:25,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:25,684 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:25,685 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:25,686 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:25,688 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:25,689 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 empty. 2023-08-23 10:53:25,690 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:25,690 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-23 10:53:25,704 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:25,705 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4240699d79069c083bb040ccf4f51d95, NAME => 'testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 4240699d79069c083bb040ccf4f51d95, disabling compactions & flushes 2023-08-23 10:53:25,720 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. after waiting 0 ms 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:25,720 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:25,720 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:25,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:25,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788005723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788005723"}]},"ts":"1692788005723"} 2023-08-23 10:53:25,726 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:25,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:25,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788005727"}]},"ts":"1692788005727"} 2023-08-23 10:53:25,729 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-08-23 10:53:25,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:25,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:25,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:25,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:25,733 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:25,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, ASSIGN}] 2023-08-23 10:53:25,736 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, ASSIGN 2023-08-23 10:53:25,736 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37675,1692787997438; forceNewPlan=false, retain=false 2023-08-23 10:53:25,887 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:25,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:25,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788005888"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788005888"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788005888"}]},"ts":"1692788005888"} 2023-08-23 10:53:25,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:26,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4240699d79069c083bb040ccf4f51d95, NAME => 'testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:26,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:26,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:26,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,050 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,052 DEBUG [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1 2023-08-23 10:53:26,052 DEBUG [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1 2023-08-23 10:53:26,052 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4240699d79069c083bb040ccf4f51d95 columnFamilyName cf1 2023-08-23 10:53:26,053 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] regionserver.HStore(310): Store=4240699d79069c083bb040ccf4f51d95/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:26,053 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,055 DEBUG [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2 2023-08-23 10:53:26,055 DEBUG [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2 2023-08-23 10:53:26,056 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4240699d79069c083bb040ccf4f51d95 columnFamilyName cf2 2023-08-23 10:53:26,057 INFO [StoreOpener-4240699d79069c083bb040ccf4f51d95-1] regionserver.HStore(310): Store=4240699d79069c083bb040ccf4f51d95/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:26,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:26,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:26,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:26,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4240699d79069c083bb040ccf4f51d95; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63267734, jitterRate=-0.05723729729652405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:26,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:26,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95., pid=27, masterSystemTime=1692788006043 2023-08-23 10:53:26,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,072 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788006072"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788006072"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788006072"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788006072"}]},"ts":"1692788006072"} 2023-08-23 10:53:26,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-08-23 10:53:26,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 in 184 msec 2023-08-23 10:53:26,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-08-23 10:53:26,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, ASSIGN in 343 msec 2023-08-23 10:53:26,080 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:26,080 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788006080"}]},"ts":"1692788006080"} 2023-08-23 10:53:26,082 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-08-23 10:53:26,086 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:26,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 403 msec 2023-08-23 10:53:26,187 DEBUG [Listener at localhost/34585] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-23 10:53:26,190 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37138, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-23 10:53:26,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:26,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:26,201 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:26,201 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,202 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:26,202 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:26,203 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,203 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,204 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,204 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,204 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,204 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,205 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,205 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,210 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,210 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,210 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,210 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,211 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,210 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,211 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,212 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,212 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:26,213 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:26,213 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,212 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,212 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,213 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:26,213 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,213 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,213 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,213 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:53:26,215 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,216 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,216 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,216 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,216 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,216 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,229 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/a269ce61217e47939a0885e93c09dbc1 2023-08-23 10:53:26,248 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/d5f6408bd7b54e7fa136919228493499 2023-08-23 10:53:26,254 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/a269ce61217e47939a0885e93c09dbc1 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 2023-08-23 10:53:26,259 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:26,260 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/d5f6408bd7b54e7fa136919228493499 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 2023-08-23 10:53:26,266 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:26,267 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 54ms, sequenceid=6, compaction requested=false 2023-08-23 10:53:26,267 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-08-23 10:53:26,267 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:26,267 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,267 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:26,267 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:26,267 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,267 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,267 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,270 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,270 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,271 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,271 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:26,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35ca3a93[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:26,274 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:26,274 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,275 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,275 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,275 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:26,276 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,276 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:26,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:26,277 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,277 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,277 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,278 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,279 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,279 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,279 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:26,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:26,285 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:26,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:26,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:26,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:26,285 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,287 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,288 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,288 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,289 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,289 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,291 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,291 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,292 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,296 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:26,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:26,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,297 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:26,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,298 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,398 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:26,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:26,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:26,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:26,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:26,421 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:26,421 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,422 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:26,422 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:26,422 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-23 10:53:26,422 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,422 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,423 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-08-23 10:53:26,423 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:26,423 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-08-23 10:53:26,424 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,424 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,424 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,424 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:26,424 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,426 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,426 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,426 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,426 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:26,428 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:26,428 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,428 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:26,429 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,436 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,437 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,441 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/d60474b0de754eadbf3242c56d92af17 2023-08-23 10:53:26,458 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/0ac8b5e2662f46d0b0916ed057cbf93b 2023-08-23 10:53:26,464 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/d60474b0de754eadbf3242c56d92af17 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 2023-08-23 10:53:26,470 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:26,470 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/0ac8b5e2662f46d0b0916ed057cbf93b as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b 2023-08-23 10:53:26,475 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:26,477 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 49ms, sequenceid=19, compaction requested=false 2023-08-23 10:53:26,477 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:26,477 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,479 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,479 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,480 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,480 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:26,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@155b852e[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:26,483 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:26,483 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,484 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,484 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:26,485 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:26,495 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,495 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,495 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,495 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,495 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,496 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,496 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,496 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,496 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,497 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:26,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:26,502 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:26,502 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:26,502 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:26,502 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:26,502 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,505 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,505 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,506 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,508 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,508 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,509 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,512 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:26,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,513 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:26,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,513 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:26,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,613 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:26,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:26,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:26,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:26,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:26,636 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:26,636 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,637 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:26,637 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:26,638 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,638 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,640 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,640 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,640 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,640 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,641 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,641 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,643 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,643 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,647 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,648 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,648 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,649 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,649 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:26,649 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:26,649 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,649 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:26,649 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:26,650 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,665 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/83f177021b164d7e8c479fbba2939606 2023-08-23 10:53:26,682 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 2023-08-23 10:53:26,688 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/83f177021b164d7e8c479fbba2939606 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 2023-08-23 10:53:26,693 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:26,694 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 2023-08-23 10:53:26,700 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:26,701 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 52ms, sequenceid=32, compaction requested=true 2023-08-23 10:53:26,701 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:26,701 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,703 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,703 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,703 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,703 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:26,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de7291c[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:26,706 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:26,706 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,709 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,709 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,710 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:26,709 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:26,711 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,711 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,711 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,712 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,712 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:26,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:26,716 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:26,716 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:26,716 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:26,716 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:26,716 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:26,717 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,717 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,718 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,718 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,718 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,719 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,720 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,722 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,722 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,722 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,727 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:26,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,726 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:26,728 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:26,727 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,728 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,728 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,728 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,828 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:26,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:26,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:26,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:26,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:26,850 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:26,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:26,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:26,851 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,851 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,852 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,852 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,852 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,852 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,853 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,853 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,854 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:26,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,855 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,855 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:26,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:26,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,858 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:26,858 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,858 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,858 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:26,858 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:26,858 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,858 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:26,858 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:26,862 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,864 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,864 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,865 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,865 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,865 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,870 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/004aba3ce0434cdc8b63f24b33344a07 2023-08-23 10:53:26,886 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/79aca82ba1b74cceb683f8558b0a4876 2023-08-23 10:53:26,891 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/004aba3ce0434cdc8b63f24b33344a07 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 2023-08-23 10:53:26,897 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:26,898 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/79aca82ba1b74cceb683f8558b0a4876 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 2023-08-23 10:53:26,904 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:26,905 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 47ms, sequenceid=45, compaction requested=true 2023-08-23 10:53:26,905 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:26,905 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:26,905 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:26,906 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:26,906 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,906 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:26,906 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:26,907 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,907 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,908 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,908 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:26,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e9cc9f0[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:26,910 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:26,910 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,911 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:26,911 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:26,911 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:26,912 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,913 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,915 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,915 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,916 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:26,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:26,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:26,919 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:26,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:26,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:26,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:26,919 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,920 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,921 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:26,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,922 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,922 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:26,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:26,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:26,925 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:26,925 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,925 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:26,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,928 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:26,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:26,929 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:26,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:26,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:26,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,929 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:26,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:26,930 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,930 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:26,930 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:26,930 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:26,930 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,030 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:27,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:27,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:27,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:27,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:27,053 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:27,053 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,053 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:27,053 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:27,053 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,053 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,054 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,055 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,055 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:27,054 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,058 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,057 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,058 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,058 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,058 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:27,058 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:27,058 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:27,058 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:27,058 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:27,060 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,060 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,061 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,061 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,061 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,061 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,070 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/5201ca41df9d4fb8bd939aa0df435545 2023-08-23 10:53:27,084 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/bc3c866d680d42f294ef0b3ced14e7ac 2023-08-23 10:53:27,090 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/5201ca41df9d4fb8bd939aa0df435545 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 2023-08-23 10:53:27,095 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:27,096 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/bc3c866d680d42f294ef0b3ced14e7ac as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac 2023-08-23 10:53:27,102 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:27,103 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 45ms, sequenceid=58, compaction requested=true 2023-08-23 10:53:27,103 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:27,103 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,103 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,105 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,105 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,106 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,106 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:27,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ae67812[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:27,108 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:27,108 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,109 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,109 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:27,111 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,111 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,111 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:27,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:27,115 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:27,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:27,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:27,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:27,115 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:27,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,118 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,118 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:27,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,131 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,131 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:27,132 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,133 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,133 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,133 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,233 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:27,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:27,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:27,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:27,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-08-23 10:53:27,258 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-08-23 10:53:27,258 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,258 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-08-23 10:53:27,258 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-08-23 10:53:27,258 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,258 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,260 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,260 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,260 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,260 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,263 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-08-23 10:53:27,263 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,263 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,263 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:27,263 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:27,263 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:27,264 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. started... 2023-08-23 10:53:27,264 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,266 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,267 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,267 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,282 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 2023-08-23 10:53:27,310 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/60aebe0a23164eb2a5d36064d3ec46d9 2023-08-23 10:53:27,317 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 2023-08-23 10:53:27,323 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:27,324 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/60aebe0a23164eb2a5d36064d3ec46d9 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 2023-08-23 10:53:27,331 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:27,332 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 68ms, sequenceid=71, compaction requested=true 2023-08-23 10:53:27,332 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:27,332 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:27,332 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:27,333 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:27,333 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,333 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-08-23 10:53:27,333 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-08-23 10:53:27,334 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,334 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,335 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,335 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-08-23 10:53:27,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43ddfe95[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:27,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-08-23 10:53:27,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,338 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,339 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:27,339 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-08-23 10:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-08-23 10:53:27,346 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-08-23 10:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-08-23 10:53:27,349 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-08-23 10:53:27,350 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:27,350 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:27,350 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-08-23 10:53:27,350 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,352 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,352 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,353 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,353 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:27,355 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-08-23 10:53:27,355 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,356 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-08-23 10:53:27,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:27,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-08-23 10:53:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-08-23 10:53:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:27,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:27,360 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:27,460 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:27,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:27,725 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-23 10:53:27,726 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:27,727 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:27,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:27,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:27,978 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,979 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:27,979 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:27,979 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:27,981 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-08-23 10:53:27,982 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:27,982 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:27,982 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:27,982 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:27,983 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:27,986 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-23 10:53:27,987 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:27,987 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:27,988 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:27,988 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:27,988 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:27,989 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:27,989 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:27,990 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:27,990 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:27,990 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,991 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:27,991 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,991 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,991 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,991 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-23 10:53:27,992 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:27,993 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:27,997 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788037997 2023-08-23 10:53:27,998 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:27,998 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:28,003 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,003 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,003 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:28,004 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788068004 2023-08-23 10:53:28,004 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788128004 2023-08-23 10:53:28,004 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:28,004 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:28,006 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:28,006 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:28,007 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:28,254 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,254 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:28,254 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:28,254 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:28,255 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:28,255 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:28,255 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:28,255 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:28,255 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:28,256 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:28,258 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-23 10:53:28,258 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:28,259 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:28,259 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:28,259 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:28,260 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,260 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:28,260 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,260 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,261 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:28,262 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,262 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:28,262 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,262 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,262 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,262 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-23 10:53:28,263 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:28,264 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:28,265 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788038265 2023-08-23 10:53:28,266 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:28,267 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,269 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,270 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,270 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:28,270 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788068270 2023-08-23 10:53:28,270 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788128270 2023-08-23 10:53:28,270 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:53:28,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:28,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN}] 2023-08-23 10:53:28,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:28,273 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:28,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:28,520 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,521 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:28,521 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:28,521 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:28,522 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:28,522 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:28,522 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:28,522 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:28,522 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:28,523 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:28,525 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-23 10:53:28,525 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:28,525 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:28,526 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:28,526 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:28,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:28,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,528 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:28,528 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,528 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:28,528 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,528 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,528 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,528 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:28,529 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:28,530 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:28,530 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788038530 2023-08-23 10:53:28,530 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:28,532 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,535 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,535 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,535 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:28,535 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788068535 2023-08-23 10:53:28,536 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788128536 2023-08-23 10:53:28,536 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:28,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN}] 2023-08-23 10:53:28,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:28,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:28,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:28,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788008539"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788008539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788008539"}]},"ts":"1692788008539"} 2023-08-23 10:53:28,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:28,541 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:28,541 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:28,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:28,541 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:28,786 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,786 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:28,786 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:28,786 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:28,787 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:28,787 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:28,787 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:28,788 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:28,788 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:28,788 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:28,791 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:28,791 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:28,791 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:28,792 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:28,792 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:28,793 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:28,793 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,794 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:28,794 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,794 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:28,795 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:28,796 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,796 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:28,796 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:28,796 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,796 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,796 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:28,796 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:28,797 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:28,797 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-23 10:53:28,797 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:28,798 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:28,798 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95 to restore RIT 2023-08-23 10:53:28,798 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788038798 2023-08-23 10:53:28,799 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:28,799 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:28,803 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,803 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:28,803 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:28,803 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788068803 2023-08-23 10:53:28,803 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788128803 2023-08-23 10:53:28,804 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:53:28,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:28,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788008804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788008804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788008804"}]},"ts":"1692788008804"} 2023-08-23 10:53:28,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:28,821 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:28,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:28,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4240699d79069c083bb040ccf4f51d95, disabling compactions & flushes 2023-08-23 10:53:28,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:28,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:28,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. after waiting 0 ms 2023-08-23 10:53:28,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:28,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4240699d79069c083bb040ccf4f51d95 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:53:28,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/590a9860c3294f67b440d771cb948039 2023-08-23 10:53:29,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/ebd0d2243dae4ec6ae211d257bf64724 2023-08-23 10:53:29,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf1/590a9860c3294f67b440d771cb948039 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 2023-08-23 10:53:29,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:29,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/.tmp/cf2/ebd0d2243dae4ec6ae211d257bf64724 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 2023-08-23 10:53:29,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:29,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4240699d79069c083bb040ccf4f51d95 in 50ms, sequenceid=82, compaction requested=true 2023-08-23 10:53:29,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:53:29,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:29,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. 2023-08-23 10:53:29,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4240699d79069c083bb040ccf4f51d95: 2023-08-23 10:53:29,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED 2023-08-23 10:53:29,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788009057"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788009057"}]},"ts":"1692788009057"} 2023-08-23 10:53:29,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:29,059 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:29,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:29,304 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,304 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:29,304 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:29,304 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:29,304 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:29,305 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:29,305 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:29,305 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:29,305 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:29,306 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:29,307 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:29,307 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-23 10:53:29,307 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-23 10:53:29,308 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:29,308 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:29,308 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:29,308 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:29,309 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,309 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:29,309 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,310 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,310 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:29,311 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,311 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,311 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:29,311 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,311 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:29,311 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:29,312 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:29,313 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:29,313 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95 to restore RIT 2023-08-23 10:53:29,313 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788039313 2023-08-23 10:53:29,314 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,314 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:29,316 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,317 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,317 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:29,317 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788069317 2023-08-23 10:53:29,317 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788129317 2023-08-23 10:53:29,317 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:29,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788009317"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788009317"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788009317"}]},"ts":"1692788009317"} 2023-08-23 10:53:29,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:29,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-08-23 10:53:29,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 in 498 msec 2023-08-23 10:53:29,322 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:29,323 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:29,323 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:29,567 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,567 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:29,567 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:29,568 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:29,568 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:29,568 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:29,568 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:29,568 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:29,568 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:29,569 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:29,570 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:29,571 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-23 10:53:29,571 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:29,571 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:29,572 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:29,572 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:29,572 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:29,573 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,573 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:29,573 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,573 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,574 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:29,574 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,575 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:29,575 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,575 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,575 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,575 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:29,575 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:29,575 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-23 10:53:29,576 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:29,576 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:29,576 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95 to restore RIT 2023-08-23 10:53:29,577 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788039577 2023-08-23 10:53:29,578 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:29,578 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,580 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,581 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,581 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:29,581 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788069581 2023-08-23 10:53:29,581 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788129581 2023-08-23 10:53:29,581 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:29,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,583 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:29,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-08-23 10:53:29,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN in 1.0470 sec 2023-08-23 10:53:29,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:29,590 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:29,591 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:29,832 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:29,832 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:29,833 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:29,834 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:29,834 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:29,834 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:29,835 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:29,835 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:29,835 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:29,835 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:29,836 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,836 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:29,836 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,837 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:29,837 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:29,838 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:29,838 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:29,839 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:29,839 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:29,839 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788039839 2023-08-23 10:53:29,841 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,841 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:29,844 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,844 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:29,844 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:29,844 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788069844 2023-08-23 10:53:29,844 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788129844 2023-08-23 10:53:29,845 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:29,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:29,850 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:29,860 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4240699d79069c083bb040ccf4f51d95, threads=14 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,861 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,862 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,862 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,862 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,862 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,862 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,863 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,863 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:29,986 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:53:30,065 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,089 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,106 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,107 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,129 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,132 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,135 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,135 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,137 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,149 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,157 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,158 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,164 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,560 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,561 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4240699d79069c083bb040ccf4f51d95 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:30,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:30,598 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:30,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:30,845 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,845 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:30,845 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:30,845 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:30,847 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:30,847 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:30,847 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:30,847 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:30,847 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:30,848 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:30,849 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-23 10:53:30,849 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:30,849 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:30,849 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:30,850 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:30,850 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:30,851 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:30,851 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:30,851 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:30,852 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:30,852 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:30,852 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:30,853 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,853 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:30,853 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:30,853 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:30,853 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,854 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,854 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,854 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:30,854 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:30,854 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:30,854 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:30,855 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:30,855 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788040855 2023-08-23 10:53:30,857 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:30,857 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:30,859 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:30,859 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:30,859 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:30,859 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788070859 2023-08-23 10:53:30,859 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788130859 2023-08-23 10:53:30,859 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:30,869 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4240699d79069c083bb040ccf4f51d95, threads=14 2023-08-23 10:53:30,870 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,870 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,870 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,870 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,871 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,871 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,871 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,871 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,872 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,872 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,872 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,872 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,873 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:30,873 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,108 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,108 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,108 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,113 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,117 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,117 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,128 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,141 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,149 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,161 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,168 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,545 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,550 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,560 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 for region: 4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:31,560 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4240699d79069c083bb040ccf4f51d95 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:31,599 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:31,604 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:31,606 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:31,606 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:31,606 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:31,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:31,609 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,610 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:31,610 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:31,610 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:31,610 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:31,610 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:31,611 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:31,611 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:31,611 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:31,611 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:31,612 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-23 10:53:31,612 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:31,612 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:31,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:31,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:31,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:31,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:31,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:31,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,615 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:31,615 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,615 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:31,615 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:31,615 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:31,615 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,615 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,616 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,616 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:31,616 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:31,616 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:31,616 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:31,617 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:31,617 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788041617 2023-08-23 10:53:31,618 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,618 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:31,620 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:31,621 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:31,621 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:31,621 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788071621 2023-08-23 10:53:31,621 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788131621 2023-08-23 10:53:31,621 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:31,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:31,626 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:31,626 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:31,626 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:31,871 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:31,871 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:31,871 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:31,871 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:31,873 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:31,873 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:31,873 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:31,873 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:31,873 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:31,874 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:31,875 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-23 10:53:31,875 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:31,875 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:31,875 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:31,875 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:31,876 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:31,876 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:31,876 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,876 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:31,877 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,877 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:31,877 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:31,878 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,878 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,878 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4240699d79069c083bb040ccf4f51d95, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:31,878 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4240699d79069c083bb040ccf4f51d95, server=jenkins-hbase4.apache.org,37675,1692787997438 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:31,878 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:31,879 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:31,879 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:31,880 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788041880 2023-08-23 10:53:31,881 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:31,881 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:31,884 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:31,884 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=4240699d79069c083bb040ccf4f51d95, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:31,884 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:31,884 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788071884 2023-08-23 10:53:31,884 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788131884 2023-08-23 10:53:31,884 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:31,889 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788011889"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1692788011889"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1692788011889"}]},"ts":"1692788011889"} 2023-08-23 10:53:31,889 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788011889"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788011889"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788011889"}]},"ts":"1692788011889"} 2023-08-23 10:53:31,889 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788011889"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788011889"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788011889"}]},"ts":"1692788011889"} 2023-08-23 10:53:31,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, ASSIGN}] 2023-08-23 10:53:31,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, ASSIGN 2023-08-23 10:53:31,899 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37675,1692787997438; forceNewPlan=false, retain=false 2023-08-23 10:53:31,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, ASSIGN 2023-08-23 10:53:31,902 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37675,1692787997438; forceNewPlan=false, retain=false 2023-08-23 10:53:32,050 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:53:32,051 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=d8f6aa47800ed8299ad4e206a840de11, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012051"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788012051"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788012051"}]},"ts":"1692788012051"} 2023-08-23 10:53:32,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure d8f6aa47800ed8299ad4e206a840de11, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:32,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=eddadea6d5040903b4cbcb5f719e6ffb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788012057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788012057"}]},"ts":"1692788012057"} 2023-08-23 10:53:32,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure eddadea6d5040903b4cbcb5f719e6ffb, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:32,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d8f6aa47800ed8299ad4e206a840de11, NAME => 'testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.', STARTKEY => '', ENDKEY => '41'} 2023-08-23 10:53:32,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,212 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,213 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1 2023-08-23 10:53:32,213 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1 2023-08-23 10:53:32,214 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8f6aa47800ed8299ad4e206a840de11 columnFamilyName cf1 2023-08-23 10:53:32,231 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-bottom 2023-08-23 10:53:32,236 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-bottom 2023-08-23 10:53:32,242 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-bottom 2023-08-23 10:53:32,249 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-bottom 2023-08-23 10:53:32,254 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-bottom 2023-08-23 10:53:32,260 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-bottom 2023-08-23 10:53:32,267 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-bottom 2023-08-23 10:53:32,267 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(310): Store=d8f6aa47800ed8299ad4e206a840de11/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:32,267 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,268 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2 2023-08-23 10:53:32,268 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2 2023-08-23 10:53:32,269 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8f6aa47800ed8299ad4e206a840de11 columnFamilyName cf2 2023-08-23 10:53:32,285 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-bottom 2023-08-23 10:53:32,290 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-bottom 2023-08-23 10:53:32,294 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-bottom 2023-08-23 10:53:32,299 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-bottom 2023-08-23 10:53:32,304 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-bottom 2023-08-23 10:53:32,309 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-bottom 2023-08-23 10:53:32,314 DEBUG [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-bottom 2023-08-23 10:53:32,314 INFO [StoreOpener-d8f6aa47800ed8299ad4e206a840de11-1] regionserver.HStore(310): Store=d8f6aa47800ed8299ad4e206a840de11/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:32,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:32,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:32,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d8f6aa47800ed8299ad4e206a840de11; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68900233, jitterRate=0.026693478226661682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:32,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d8f6aa47800ed8299ad4e206a840de11: 2023-08-23 10:53:32,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11., pid=33, masterSystemTime=1692788012206 2023-08-23 10:53:32,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:32,322 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:32,322 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:32,322 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,323 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,323 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1912): d8f6aa47800ed8299ad4e206a840de11/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:32,323 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1912): d8f6aa47800ed8299ad4e206a840de11/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:32,323 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d8f6aa47800ed8299ad4e206a840de11/cf1 in testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,323 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d8f6aa47800ed8299ad4e206a840de11/cf2 in testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,323 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/.tmp, totalSize=34.9 K 2023-08-23 10:53:32,323 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/.tmp, totalSize=34.9 K 2023-08-23 10:53:32,324 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788006191 2023-08-23 10:53:32,324 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788006191 2023-08-23 10:53:32,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:32,324 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788006400 2023-08-23 10:53:32,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eddadea6d5040903b4cbcb5f719e6ffb, NAME => 'testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.', STARTKEY => '41', ENDKEY => ''} 2023-08-23 10:53:32,325 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788006400 2023-08-23 10:53:32,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:32,325 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788006616 2023-08-23 10:53:32,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:32,325 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788006616 2023-08-23 10:53:32,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=d8f6aa47800ed8299ad4e206a840de11, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,326 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788006830 2023-08-23 10:53:32,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012325"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788012325"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788012325"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788012325"}]},"ts":"1692788012325"} 2023-08-23 10:53:32,326 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788006830 2023-08-23 10:53:32,327 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788007032 2023-08-23 10:53:32,327 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788007032 2023-08-23 10:53:32,327 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788007235 2023-08-23 10:53:32,328 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788007235 2023-08-23 10:53:32,329 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788007463 2023-08-23 10:53:32,329 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788007463 2023-08-23 10:53:32,330 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-08-23 10:53:32,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure d8f6aa47800ed8299ad4e206a840de11, server=jenkins-hbase4.apache.org,37675,1692787997438 in 274 msec 2023-08-23 10:53:32,331 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1 2023-08-23 10:53:32,331 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1 2023-08-23 10:53:32,332 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eddadea6d5040903b4cbcb5f719e6ffb columnFamilyName cf1 2023-08-23 10:53:32,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, ASSIGN in 437 msec 2023-08-23 10:53:32,347 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] throttle.PressureAwareThroughputController(145): d8f6aa47800ed8299ad4e206a840de11#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:32,352 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-top 2023-08-23 10:53:32,358 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-top 2023-08-23 10:53:32,359 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] throttle.PressureAwareThroughputController(145): d8f6aa47800ed8299ad4e206a840de11#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:32,368 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-top 2023-08-23 10:53:32,373 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-top 2023-08-23 10:53:32,382 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-top 2023-08-23 10:53:32,383 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/.tmp/cf2/6b37e1dd5a164acebdd1ac898bc572a8 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/6b37e1dd5a164acebdd1ac898bc572a8 2023-08-23 10:53:32,386 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-top 2023-08-23 10:53:32,396 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-top 2023-08-23 10:53:32,396 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(310): Store=eddadea6d5040903b4cbcb5f719e6ffb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:32,397 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,397 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d8f6aa47800ed8299ad4e206a840de11/cf2 of d8f6aa47800ed8299ad4e206a840de11 into 6b37e1dd5a164acebdd1ac898bc572a8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:32,398 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d8f6aa47800ed8299ad4e206a840de11: 2023-08-23 10:53:32,398 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11., storeName=d8f6aa47800ed8299ad4e206a840de11/cf2, priority=9, startTime=1692788012321; duration=0sec 2023-08-23 10:53:32,398 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,399 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2 2023-08-23 10:53:32,399 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2 2023-08-23 10:53:32,399 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eddadea6d5040903b4cbcb5f719e6ffb columnFamilyName cf2 2023-08-23 10:53:32,404 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/.tmp/cf1/bbab640d24cc46eba3cdddb3b36e7255 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/bbab640d24cc46eba3cdddb3b36e7255 2023-08-23 10:53:32,411 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d8f6aa47800ed8299ad4e206a840de11/cf1 of d8f6aa47800ed8299ad4e206a840de11 into bbab640d24cc46eba3cdddb3b36e7255(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:32,411 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d8f6aa47800ed8299ad4e206a840de11: 2023-08-23 10:53:32,411 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11., storeName=d8f6aa47800ed8299ad4e206a840de11/cf1, priority=9, startTime=1692788012321; duration=0sec 2023-08-23 10:53:32,411 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,422 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-top 2023-08-23 10:53:32,426 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-top 2023-08-23 10:53:32,430 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-top 2023-08-23 10:53:32,435 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-top 2023-08-23 10:53:32,439 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-top 2023-08-23 10:53:32,446 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-top 2023-08-23 10:53:32,450 DEBUG [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-top 2023-08-23 10:53:32,451 INFO [StoreOpener-eddadea6d5040903b4cbcb5f719e6ffb-1] regionserver.HStore(310): Store=eddadea6d5040903b4cbcb5f719e6ffb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:32,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:32,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eddadea6d5040903b4cbcb5f719e6ffb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72648165, jitterRate=0.08254201710224152}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:32,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eddadea6d5040903b4cbcb5f719e6ffb: 2023-08-23 10:53:32,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb., pid=34, masterSystemTime=1692788012206 2023-08-23 10:53:32,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:32,459 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:32,459 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:32,459 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,459 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,460 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1912): eddadea6d5040903b4cbcb5f719e6ffb/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:32,460 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1912): eddadea6d5040903b4cbcb5f719e6ffb/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:32,460 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eddadea6d5040903b4cbcb5f719e6ffb/cf1 in testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,460 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegion(2259): Starting compaction of eddadea6d5040903b4cbcb5f719e6ffb/cf2 in testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,460 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/.tmp, totalSize=34.9 K 2023-08-23 10:53:32,460 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/.tmp, totalSize=34.9 K 2023-08-23 10:53:32,461 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788006191 2023-08-23 10:53:32,461 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788006191 2023-08-23 10:53:32,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,462 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788006400 2023-08-23 10:53:32,462 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788006400 2023-08-23 10:53:32,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=eddadea6d5040903b4cbcb5f719e6ffb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,463 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012462"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788012462"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788012462"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788012462"}]},"ts":"1692788012462"} 2023-08-23 10:53:32,463 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788006616 2023-08-23 10:53:32,463 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788006616 2023-08-23 10:53:32,463 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788006830 2023-08-23 10:53:32,464 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788006830 2023-08-23 10:53:32,464 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788007032 2023-08-23 10:53:32,465 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788007032 2023-08-23 10:53:32,465 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting 60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788007235 2023-08-23 10:53:32,465 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788007235 2023-08-23 10:53:32,466 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] compactions.Compactor(207): Compacting ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788007463 2023-08-23 10:53:32,466 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] compactions.Compactor(207): Compacting 590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788007463 2023-08-23 10:53:32,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-08-23 10:53:32,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure eddadea6d5040903b4cbcb5f719e6ffb, server=jenkins-hbase4.apache.org,37675,1692787997438 in 406 msec 2023-08-23 10:53:32,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-08-23 10:53:32,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, ASSIGN in 574 msec 2023-08-23 10:53:32,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb in 4.7450 sec 2023-08-23 10:53:32,484 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] throttle.PressureAwareThroughputController(145): eddadea6d5040903b4cbcb5f719e6ffb#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:32,484 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] throttle.PressureAwareThroughputController(145): eddadea6d5040903b4cbcb5f719e6ffb#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:32,503 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/.tmp/cf2/c251e54fdf7a4562b51368189c921902 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/c251e54fdf7a4562b51368189c921902 2023-08-23 10:53:32,506 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/.tmp/cf1/5f4503f84bcd4c919ccfca30f1036e6e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5f4503f84bcd4c919ccfca30f1036e6e 2023-08-23 10:53:32,509 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eddadea6d5040903b4cbcb5f719e6ffb/cf2 of eddadea6d5040903b4cbcb5f719e6ffb into c251e54fdf7a4562b51368189c921902(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:32,510 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for eddadea6d5040903b4cbcb5f719e6ffb: 2023-08-23 10:53:32,510 INFO [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb., storeName=eddadea6d5040903b4cbcb5f719e6ffb/cf2, priority=9, startTime=1692788012459; duration=0sec 2023-08-23 10:53:32,510 DEBUG [RS:0;jenkins-hbase4:37675-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,512 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eddadea6d5040903b4cbcb5f719e6ffb/cf1 of eddadea6d5040903b4cbcb5f719e6ffb into 5f4503f84bcd4c919ccfca30f1036e6e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:32,512 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eddadea6d5040903b4cbcb5f719e6ffb: 2023-08-23 10:53:32,512 INFO [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb., storeName=eddadea6d5040903b4cbcb5f719e6ffb/cf1, priority=9, startTime=1692788012458; duration=0sec 2023-08-23 10:53:32,512 DEBUG [RS:0;jenkins-hbase4:37675-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:32,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37675] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:37138 deadline: 1692788072636, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95. is not online on jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,749 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:53:32,758 INFO [Listener at localhost/34585] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-23 10:53:32,760 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:53:32,778 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:32,779 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-08-23 10:53:32,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-08-23 10:53:32,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:32,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-23 10:53:32,785 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788012785"}]},"ts":"1692788012785"} 2023-08-23 10:53:32,786 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-08-23 10:53:32,812 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-08-23 10:53:32,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, UNASSIGN}] 2023-08-23 10:53:32,814 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, UNASSIGN 2023-08-23 10:53:32,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=eddadea6d5040903b4cbcb5f719e6ffb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012815"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788012815"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788012815"}]},"ts":"1692788012815"} 2023-08-23 10:53:32,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure eddadea6d5040903b4cbcb5f719e6ffb, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:32,820 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, UNASSIGN 2023-08-23 10:53:32,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=d8f6aa47800ed8299ad4e206a840de11, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:32,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788012820"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788012820"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788012820"}]},"ts":"1692788012820"} 2023-08-23 10:53:32,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure d8f6aa47800ed8299ad4e206a840de11, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:32,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-23 10:53:32,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:32,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eddadea6d5040903b4cbcb5f719e6ffb, disabling compactions & flushes 2023-08-23 10:53:32,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. after waiting 0 ms 2023-08-23 10:53:32,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:32,982 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-top] to archive 2023-08-23 10:53:32,984 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:32,986 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,988 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,989 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,991 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,992 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,993 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,995 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:32,997 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-top] to archive 2023-08-23 10:53:32,998 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:33,000 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,001 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,003 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,004 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,005 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,007 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,008 DEBUG [StoreCloser-testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:53:33,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:33,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb. 2023-08-23 10:53:33,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eddadea6d5040903b4cbcb5f719e6ffb: 2023-08-23 10:53:33,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:33,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:33,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d8f6aa47800ed8299ad4e206a840de11, disabling compactions & flushes 2023-08-23 10:53:33,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:33,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:33,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. after waiting 0 ms 2023-08-23 10:53:33,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:33,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=eddadea6d5040903b4cbcb5f719e6ffb, regionState=CLOSED 2023-08-23 10:53:33,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788013015"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788013015"}]},"ts":"1692788013015"} 2023-08-23 10:53:33,016 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039-bottom] to archive 2023-08-23 10:53:33,017 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:33,019 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-08-23 10:53:33,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure eddadea6d5040903b4cbcb5f719e6ffb, server=jenkins-hbase4.apache.org,37675,1692787997438 in 200 msec 2023-08-23 10:53:33,020 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eddadea6d5040903b4cbcb5f719e6ffb, UNASSIGN in 206 msec 2023-08-23 10:53:33,022 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,023 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,025 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,026 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,027 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,030 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724-bottom] to archive 2023-08-23 10:53:33,030 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:33,032 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,033 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,034 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,036 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,037 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,038 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,039 DEBUG [StoreCloser-testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:53:33,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:33,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11. 2023-08-23 10:53:33,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d8f6aa47800ed8299ad4e206a840de11: 2023-08-23 10:53:33,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:33,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=d8f6aa47800ed8299ad4e206a840de11, regionState=CLOSED 2023-08-23 10:53:33,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788013046"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788013046"}]},"ts":"1692788013046"} 2023-08-23 10:53:33,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-08-23 10:53:33,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure d8f6aa47800ed8299ad4e206a840de11, server=jenkins-hbase4.apache.org,37675,1692787997438 in 226 msec 2023-08-23 10:53:33,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-08-23 10:53:33,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=d8f6aa47800ed8299ad4e206a840de11, UNASSIGN in 237 msec 2023-08-23 10:53:33,053 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788013053"}]},"ts":"1692788013053"} 2023-08-23 10:53:33,054 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-08-23 10:53:33,056 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-08-23 10:53:33,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 276 msec 2023-08-23 10:53:33,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-08-23 10:53:33,087 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-08-23 10:53:33,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-08-23 10:53:33,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,091 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,092 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-23 10:53:33,100 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,100 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:33,100 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:33,103 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/recovered.edits] 2023-08-23 10:53:33,103 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/recovered.edits] 2023-08-23 10:53:33,104 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/recovered.edits] 2023-08-23 10:53:33,107 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/bbab640d24cc46eba3cdddb3b36e7255 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf1/bbab640d24cc46eba3cdddb3b36e7255 2023-08-23 10:53:33,108 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5f4503f84bcd4c919ccfca30f1036e6e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5f4503f84bcd4c919ccfca30f1036e6e 2023-08-23 10:53:33,108 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/004aba3ce0434cdc8b63f24b33344a07 2023-08-23 10:53:33,110 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/26c8d62c6b9b4b2cbb79b71ccedad324 2023-08-23 10:53:33,111 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/6b37e1dd5a164acebdd1ac898bc572a8 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/cf2/6b37e1dd5a164acebdd1ac898bc572a8 2023-08-23 10:53:33,111 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/c251e54fdf7a4562b51368189c921902 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/cf2/c251e54fdf7a4562b51368189c921902 2023-08-23 10:53:33,111 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/5201ca41df9d4fb8bd939aa0df435545 2023-08-23 10:53:33,114 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/590a9860c3294f67b440d771cb948039 2023-08-23 10:53:33,115 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/.regioninfo to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/.regioninfo 2023-08-23 10:53:33,115 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/.regioninfo to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/.regioninfo 2023-08-23 10:53:33,115 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/83f177021b164d7e8c479fbba2939606 2023-08-23 10:53:33,117 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/a269ce61217e47939a0885e93c09dbc1 2023-08-23 10:53:33,119 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,119 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf1/d60474b0de754eadbf3242c56d92af17 2023-08-23 10:53:33,120 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/004aba3ce0434cdc8b63f24b33344a07.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,121 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,123 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/26c8d62c6b9b4b2cbb79b71ccedad324.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,123 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,124 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/0ac8b5e2662f46d0b0916ed057cbf93b 2023-08-23 10:53:33,124 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/5201ca41df9d4fb8bd939aa0df435545.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,124 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,126 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/60aebe0a23164eb2a5d36064d3ec46d9 2023-08-23 10:53:33,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/590a9860c3294f67b440d771cb948039.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,127 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,129 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/83f177021b164d7e8c479fbba2939606.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,129 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/79aca82ba1b74cceb683f8558b0a4876 2023-08-23 10:53:33,129 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,131 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/bc3c866d680d42f294ef0b3ced14e7ac 2023-08-23 10:53:33,131 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/a269ce61217e47939a0885e93c09dbc1.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,131 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,133 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf1/d60474b0de754eadbf3242c56d92af17.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,134 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/d5f6408bd7b54e7fa136919228493499 2023-08-23 10:53:33,142 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ebd0d2243dae4ec6ae211d257bf64724 2023-08-23 10:53:33,144 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/cf2/ff4c5dcf2e02400b9fb920e6b01cf347 2023-08-23 10:53:33,145 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,145 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/0ac8b5e2662f46d0b0916ed057cbf93b.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,148 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,148 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/60aebe0a23164eb2a5d36064d3ec46d9.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95/recovered.edits/85.seqid 2023-08-23 10:53:33,150 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/79aca82ba1b74cceb683f8558b0a4876.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,151 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,151 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,152 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/bc3c866d680d42f294ef0b3ced14e7ac.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,153 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,153 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/d5f6408bd7b54e7fa136919228493499.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,155 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,155 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ebd0d2243dae4ec6ae211d257bf64724.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,157 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/d8f6aa47800ed8299ad4e206a840de11/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,157 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/eddadea6d5040903b4cbcb5f719e6ffb/cf2/ff4c5dcf2e02400b9fb920e6b01cf347.4240699d79069c083bb040ccf4f51d95 2023-08-23 10:53:33,160 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11/recovered.edits/90.seqid 2023-08-23 10:53:33,161 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb/recovered.edits/90.seqid 2023-08-23 10:53:33,162 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/d8f6aa47800ed8299ad4e206a840de11 2023-08-23 10:53:33,162 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitWithoutPONR/eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:33,162 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-08-23 10:53:33,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,167 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-08-23 10:53:33,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-08-23 10:53:33,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,173 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-08-23 10:53:33,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788013173"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:33,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788013173"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:33,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788013173"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:33,175 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:53:33,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4240699d79069c083bb040ccf4f51d95, NAME => 'testSplitWithoutPONR,,1692788005682.4240699d79069c083bb040ccf4f51d95.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d8f6aa47800ed8299ad4e206a840de11, NAME => 'testSplitWithoutPONR,,1692788007726.d8f6aa47800ed8299ad4e206a840de11.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => eddadea6d5040903b4cbcb5f719e6ffb, NAME => 'testSplitWithoutPONR,41,1692788007726.eddadea6d5040903b4cbcb5f719e6ffb.', STARTKEY => '41', ENDKEY => ''}] 2023-08-23 10:53:33,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-08-23 10:53:33,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788013175"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:33,177 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-08-23 10:53:33,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:33,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 91 msec 2023-08-23 10:53:33,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-08-23 10:53:33,194 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-08-23 10:53:33,211 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=407 (was 396) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:49706 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:49716 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:58894 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:45192 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:37675-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:49328 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:58774 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:58828 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:59460 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7797637d-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:39250 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:45784 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:54290 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:39200 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:49730 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:58910 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:55844 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1683557519_17 at /127.0.0.1:39192 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 815) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=337 (was 304) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=10975 (was 10977) 2023-08-23 10:53:33,229 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=407, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=171, AvailableMemoryMB=10974 2023-08-23 10:53:33,229 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:33,232 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:33,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:33,234 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:33,236 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:33,237 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:33,239 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,240 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 empty. 2023-08-23 10:53:33,240 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-23 10:53:33,256 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:33,258 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 84b37956ab3c45570fda115a32fc8299, NAME => 'testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:33,267 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:33,267 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 84b37956ab3c45570fda115a32fc8299, disabling compactions & flushes 2023-08-23 10:53:33,267 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,268 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,268 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. after waiting 0 ms 2023-08-23 10:53:33,268 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,268 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,268 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:33,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:33,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788013271"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788013271"}]},"ts":"1692788013271"} 2023-08-23 10:53:33,274 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:33,275 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:33,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788013275"}]},"ts":"1692788013275"} 2023-08-23 10:53:33,276 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-23 10:53:33,280 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:33,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:33,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:33,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:33,282 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:33,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN}] 2023-08-23 10:53:33,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:33,285 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:53:33,435 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:33,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,437 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788013436"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788013436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788013436"}]},"ts":"1692788013436"} 2023-08-23 10:53:33,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:33,591 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,591 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-23 10:53:33,593 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38518, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-23 10:53:33,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 84b37956ab3c45570fda115a32fc8299, NAME => 'testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:33,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:33,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:33,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,603 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,604 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1 2023-08-23 10:53:33,604 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1 2023-08-23 10:53:33,604 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b37956ab3c45570fda115a32fc8299 columnFamilyName cf1 2023-08-23 10:53:33,605 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(310): Store=84b37956ab3c45570fda115a32fc8299/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:33,605 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,606 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2 2023-08-23 10:53:33,606 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2 2023-08-23 10:53:33,607 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b37956ab3c45570fda115a32fc8299 columnFamilyName cf2 2023-08-23 10:53:33,607 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(310): Store=84b37956ab3c45570fda115a32fc8299/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:33,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:33,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:33,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:33,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 84b37956ab3c45570fda115a32fc8299; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74529860, jitterRate=0.11058145761489868}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:33,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:33,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299., pid=43, masterSystemTime=1692788013591 2023-08-23 10:53:33,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,621 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788013620"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788013620"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788013620"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788013620"}]},"ts":"1692788013620"} 2023-08-23 10:53:33,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-08-23 10:53:33,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 in 184 msec 2023-08-23 10:53:33,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-08-23 10:53:33,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN in 342 msec 2023-08-23 10:53:33,627 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:33,627 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788013627"}]},"ts":"1692788013627"} 2023-08-23 10:53:33,629 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-23 10:53:33,636 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:33,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 403 msec 2023-08-23 10:53:33,737 DEBUG [Listener at localhost/34585] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-08-23 10:53:33,742 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38534, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-08-23 10:53:33,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:33,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:33,752 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:33,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:33,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:33,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,754 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,755 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:33,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,756 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:33,758 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,759 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,759 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:33,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:33,760 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:33,761 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,761 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:33,762 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:53:33,763 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,764 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,764 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:33,778 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 2023-08-23 10:53:33,800 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/c427a71642ba43aca96fd823cb32b86d 2023-08-23 10:53:33,805 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 2023-08-23 10:53:33,810 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:33,811 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/c427a71642ba43aca96fd823cb32b86d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d 2023-08-23 10:53:33,815 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:33,816 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 54ms, sequenceid=6, compaction requested=false 2023-08-23 10:53:33,816 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-08-23 10:53:33,817 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:33,817 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,817 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:33,817 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:33,818 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,818 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:33,818 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:33,819 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,819 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:33,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:33,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:33,822 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:33,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1726903b[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:33,822 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:33,825 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:33,825 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:33,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,825 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:33,825 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:33,826 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:33,826 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:33,825 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:33,826 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:33,826 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:33,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:33,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:33,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:33,828 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:33,828 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:33,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:33,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:33,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:33,832 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:33,832 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:33,832 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:33,832 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:33,832 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:33,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,834 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,835 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,835 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,836 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,838 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:33,838 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:33,838 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:33,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:33,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:33,843 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:33,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,842 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:33,843 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:33,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,844 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:33,844 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,844 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:33,844 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,844 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,944 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:33,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:33,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:33,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:33,965 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:33,965 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,965 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:33,965 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:33,966 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,966 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,967 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,967 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,967 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,967 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:33,968 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:33,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:33,968 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:33,969 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,969 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:33,970 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:33,970 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:33,970 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:33,970 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:33,971 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,972 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,972 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,972 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:33,973 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:33,973 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:33,987 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/36a6c20ebe1a411592a349d8bc0b4bdf 2023-08-23 10:53:34,005 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/32d518f51c0a4c7ba83cf81a44b39887 2023-08-23 10:53:34,010 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/36a6c20ebe1a411592a349d8bc0b4bdf as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf 2023-08-23 10:53:34,015 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:34,016 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/32d518f51c0a4c7ba83cf81a44b39887 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 2023-08-23 10:53:34,021 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:34,022 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 52ms, sequenceid=19, compaction requested=false 2023-08-23 10:53:34,022 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:34,022 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,022 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:34,022 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:34,022 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,022 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,022 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,025 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,025 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,025 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,025 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:34,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f7d0866[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:34,027 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:34,027 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,029 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,029 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:34,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,035 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,035 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,035 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,036 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:34,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:34,039 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:34,039 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:34,039 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:34,039 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:34,039 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,041 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,041 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,042 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,042 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,043 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,045 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,050 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,050 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:34,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,050 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,151 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:34,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:34,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:34,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:34,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:34,171 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,171 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,171 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:34,171 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:34,171 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,171 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,173 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,173 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,173 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,173 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:34,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:34,175 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,175 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,176 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,176 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,176 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:34,178 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,179 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,179 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,179 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,179 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,179 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,189 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/eeeef2c2015947668d988558e7e6bcf6 2023-08-23 10:53:34,202 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/ae8abafb56d942208ab2fd40791a960c 2023-08-23 10:53:34,208 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/eeeef2c2015947668d988558e7e6bcf6 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 2023-08-23 10:53:34,213 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:34,214 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/ae8abafb56d942208ab2fd40791a960c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c 2023-08-23 10:53:34,219 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:34,220 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 44ms, sequenceid=32, compaction requested=true 2023-08-23 10:53:34,220 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:34,220 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,220 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:34,220 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:34,220 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,220 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,220 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,222 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,222 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,223 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,223 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:34,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2662d573[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:34,225 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:34,225 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,226 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,226 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,227 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:34,226 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,226 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:34,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:34,227 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:34,230 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,230 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:34,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:34,234 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:34,234 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:34,234 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:34,234 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:34,234 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,236 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,237 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,237 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,238 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,239 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,241 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,241 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,248 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:34,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,249 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,349 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:34,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:34,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:34,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:34,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:34,368 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,368 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,368 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:34,368 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:34,369 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,369 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,371 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,371 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,371 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,372 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,372 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,373 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:34,374 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:34,374 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,374 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:34,374 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:34,375 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,376 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,376 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,376 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,376 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,377 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,392 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/6e52ed8c4f674ff9aced32139482eff3 2023-08-23 10:53:34,410 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 2023-08-23 10:53:34,416 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/6e52ed8c4f674ff9aced32139482eff3 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 2023-08-23 10:53:34,424 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:34,425 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 2023-08-23 10:53:34,430 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:34,431 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 57ms, sequenceid=45, compaction requested=true 2023-08-23 10:53:34,432 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:34,432 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,432 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:34,432 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:34,432 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,432 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,432 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,434 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,434 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,434 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,434 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:34,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ae7be7c[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:34,437 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:34,437 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,440 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,440 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,440 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,441 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,443 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,443 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,443 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,444 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:34,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:34,448 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:34,448 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:34,448 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:34,448 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:34,448 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:34,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,457 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,457 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,457 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,458 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,458 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,461 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,461 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,461 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,464 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,464 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,464 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,464 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,465 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,464 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:34,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,465 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,466 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:34,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,566 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:34,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:34,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:34,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:34,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:34,584 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,584 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,584 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:34,584 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:34,585 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,585 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,586 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,586 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,586 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,586 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,587 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,587 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:34,589 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,589 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,589 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:34,589 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,601 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/427e8da914bd476390bef84aaa95bcaf 2023-08-23 10:53:34,614 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 2023-08-23 10:53:34,619 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/427e8da914bd476390bef84aaa95bcaf as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf 2023-08-23 10:53:34,623 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:34,624 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 2023-08-23 10:53:34,628 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:34,629 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 40ms, sequenceid=58, compaction requested=true 2023-08-23 10:53:34,629 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:34,629 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,629 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:34,629 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:34,629 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,629 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,629 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,631 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,631 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,632 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29635053[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:34,633 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:34,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,635 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:34,640 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,641 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:34,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:34,644 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:34,644 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:34,644 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:34,644 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:34,644 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,647 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,647 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,649 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,650 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,650 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:34,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,653 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,654 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:34,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,654 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,655 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,655 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,655 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,655 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,655 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,755 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:34,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:34,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:34,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:34,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-08-23 10:53:34,774 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,774 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,774 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-08-23 10:53:34,774 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:53:34,774 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,774 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,776 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,776 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,776 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,776 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,777 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,777 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,778 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:34,779 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,779 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. started... 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:34,779 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,779 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:34,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,781 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,781 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,781 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,790 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/095a08d0d6f04fdc979670d18d5b3100 2023-08-23 10:53:34,804 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 2023-08-23 10:53:34,809 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/095a08d0d6f04fdc979670d18d5b3100 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 2023-08-23 10:53:34,814 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:34,814 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 2023-08-23 10:53:34,819 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:34,820 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 41ms, sequenceid=71, compaction requested=true 2023-08-23 10:53:34,820 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:34,820 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:34,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:34,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:34,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-08-23 10:53:34,820 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-08-23 10:53:34,822 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,822 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,822 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,822 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:34,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-08-23 10:53:34,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e2a5321[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:34,824 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:53:34,825 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,827 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:34,827 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-08-23 10:53:34,827 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:34,828 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,829 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-08-23 10:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-08-23 10:53:34,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:53:34,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:53:34,833 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-08-23 10:53:34,833 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:34,833 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:34,833 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-08-23 10:53:34,833 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,837 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:34,837 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,838 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:34,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-08-23 10:53:34,839 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,840 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,840 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:34,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-08-23 10:53:34,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-08-23 10:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:34,850 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-08-23 10:53:34,950 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:34,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:35,212 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-23 10:53:35,213 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:35,214 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,216 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:35,216 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:35,216 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:35,464 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,464 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:35,464 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:35,465 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:35,465 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-08-23 10:53:35,465 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:35,465 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:35,465 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:35,465 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:35,466 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:35,467 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-23 10:53:35,468 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:35,468 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:35,469 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:35,469 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:35,469 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,469 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:35,469 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,470 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,470 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,470 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:35,470 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:35,472 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,472 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:35,472 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,472 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,472 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,472 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:35,473 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:35,473 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:35,474 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788045474 2023-08-23 10:53:35,476 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,476 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:35,478 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:35,479 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:35,479 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:35,479 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788075479 2023-08-23 10:53:35,479 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788135479 2023-08-23 10:53:35,479 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:35,479 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:35,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:35,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:35,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:35,729 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,729 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:35,729 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:35,729 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:35,731 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:35,731 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:35,731 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:35,731 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:35,731 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:35,732 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:35,733 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-23 10:53:35,733 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:35,734 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:35,734 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:35,734 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:35,734 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,735 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:35,735 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,735 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:35,735 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,735 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:35,736 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,736 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:35,736 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:35,736 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,736 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:35,736 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,737 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,737 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,737 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:35,737 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:35,738 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:35,738 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788045738 2023-08-23 10:53:35,740 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:35,740 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,742 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:35,742 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:35,742 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:35,742 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788075742 2023-08-23 10:53:35,743 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788135743 2023-08-23 10:53:35,743 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:35,743 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:35,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN}] 2023-08-23 10:53:35,745 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:35,745 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:35,745 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:35,993 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:35,993 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:35,993 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:35,993 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:35,995 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:35,996 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:35,996 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:35,996 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:35,996 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:35,996 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:35,998 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-23 10:53:35,998 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:35,999 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:35,999 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:35,999 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:35,999 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,000 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:36,000 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,000 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,000 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,001 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:36,001 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,001 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,002 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:36,002 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,002 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,002 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,002 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,002 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,002 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:36,003 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:36,003 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:36,004 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788046004 2023-08-23 10:53:36,005 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:36,005 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,007 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:36,007 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:36,008 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:36,008 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788076008 2023-08-23 10:53:36,008 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788136008 2023-08-23 10:53:36,008 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:36,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN}] 2023-08-23 10:53:36,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:36,010 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,011 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,011 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788016011"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788016011"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788016011"}]},"ts":"1692788016011"} 2023-08-23 10:53:36,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:36,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:36,013 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:36,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:36,258 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,258 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:36,258 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:36,258 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:36,259 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:36,259 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:36,259 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:36,259 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:36,259 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:36,260 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:36,261 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:36,261 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:36,261 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:36,261 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:36,262 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:36,262 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:36,262 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,262 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:36,263 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,263 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,263 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,263 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:36,263 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,264 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,264 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:36,264 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,265 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,265 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,265 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,265 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,265 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,265 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,265 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,265 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:36,266 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:36,267 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:36,267 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:36,267 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788046267 2023-08-23 10:53:36,267 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:36,268 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,271 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:36,271 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:36,271 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:36,271 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788076271 2023-08-23 10:53:36,271 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788136271 2023-08-23 10:53:36,271 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:53:36,272 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,272 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788016272"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788016272"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788016272"}]},"ts":"1692788016272"} 2023-08-23 10:53:36,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:36,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:36,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:36,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 84b37956ab3c45570fda115a32fc8299, disabling compactions & flushes 2023-08-23 10:53:36,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:36,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:36,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. after waiting 0 ms 2023-08-23 10:53:36,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:36,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 84b37956ab3c45570fda115a32fc8299 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:53:36,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc 2023-08-23 10:53:36,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/2b0a01d941fe41448c0b22aa55d9065b 2023-08-23 10:53:36,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc 2023-08-23 10:53:36,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:36,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/2b0a01d941fe41448c0b22aa55d9065b as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b 2023-08-23 10:53:36,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:53:36,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 84b37956ab3c45570fda115a32fc8299 in 40ms, sequenceid=82, compaction requested=true 2023-08-23 10:53:36,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:53:36,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:36,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:36,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:36,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:36,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED 2023-08-23 10:53:36,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788016476"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788016476"}]},"ts":"1692788016476"} 2023-08-23 10:53:36,478 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:36,478 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:36,478 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:36,522 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,522 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:36,522 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:36,522 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:36,523 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:36,524 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:36,524 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:36,524 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:36,524 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:36,525 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:36,526 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:36,526 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-23 10:53:36,526 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-08-23 10:53:36,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:36,527 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:36,528 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:36,528 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:36,528 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,529 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:36,529 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,529 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,530 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,530 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:36,530 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,531 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,531 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:36,532 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,532 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,532 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,532 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,532 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,533 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,533 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,533 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,533 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,533 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,533 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-23 10:53:36,534 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:36,535 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:36,535 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:36,535 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788046535 2023-08-23 10:53:36,536 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,536 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:36,539 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:36,540 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:36,540 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:36,540 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788076540 2023-08-23 10:53:36,540 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788136540 2023-08-23 10:53:36,540 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:36,541 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,541 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788016540"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788016540"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788016540"}]},"ts":"1692788016540"} 2023-08-23 10:53:36,542 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:36,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-08-23 10:53:36,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 in 268 msec 2023-08-23 10:53:36,546 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:36,546 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:36,546 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:36,791 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,791 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:36,791 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:36,791 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:36,792 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:36,792 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:36,792 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:36,793 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:36,793 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:36,794 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:36,795 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:36,795 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-08-23 10:53:36,795 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:36,796 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:36,796 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:36,796 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:36,797 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:36,797 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,797 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:36,798 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,798 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:36,798 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,799 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:36,799 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,799 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:36,800 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:36,801 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,801 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,801 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:36,801 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:36,801 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,801 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,801 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,802 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:36,802 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:36,802 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-23 10:53:36,803 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:36,803 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:36,803 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:36,804 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788046804 2023-08-23 10:53:36,804 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:36,805 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,807 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:36,807 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:36,807 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:36,807 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788076807 2023-08-23 10:53:36,807 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788136807 2023-08-23 10:53:36,808 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:36,808 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:36,809 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:36,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-08-23 10:53:36,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN in 801 msec 2023-08-23 10:53:36,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:36,815 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:36,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:36,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:37,058 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,058 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:37,058 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:37,058 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:37,058 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:37,059 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:37,059 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:37,059 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:37,059 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:37,059 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:37,060 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:37,061 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:37,061 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:37,061 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:37,061 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:37,061 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:37,062 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:37,062 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,062 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:37,062 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,063 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,063 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,063 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:37,063 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,063 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,064 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:37,064 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,065 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:37,065 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:37,065 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:37,065 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:37,065 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:37,065 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:37,066 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:37,066 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:37,066 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788047066 2023-08-23 10:53:37,067 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:37,068 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,071 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:37,071 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:37,071 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:37,071 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788077071 2023-08-23 10:53:37,071 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788137071 2023-08-23 10:53:37,071 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:37,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,077 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:37,087 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=84b37956ab3c45570fda115a32fc8299, threads=14 2023-08-23 10:53:37,088 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,088 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,088 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,089 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,089 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,089 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,090 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,090 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,091 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,091 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,091 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,091 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,092 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,092 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,343 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,343 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,357 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,359 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,360 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,385 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,386 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,386 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,387 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,387 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,393 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,405 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,409 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,412 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,412 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 84b37956ab3c45570fda115a32fc8299 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:37,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:37,456 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:37,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:37,572 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:37,572 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:37,573 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:37,573 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:37,574 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:37,575 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:37,576 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,576 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:37,576 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,576 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:37,577 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,577 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:37,577 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,577 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:37,577 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:37,578 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,578 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,578 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:37,578 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:37,579 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:37,579 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:37,580 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:37,580 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788047580 2023-08-23 10:53:37,582 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:37,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:37,584 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:37,584 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:37,584 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:37,584 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788077584 2023-08-23 10:53:37,584 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788137584 2023-08-23 10:53:37,584 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:37,595 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=84b37956ab3c45570fda115a32fc8299, threads=14 2023-08-23 10:53:37,596 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,596 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,596 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,596 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,597 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,597 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,597 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,597 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,598 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,598 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,598 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,601 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,601 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,601 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,713 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:53:37,841 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,842 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,869 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,874 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,893 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,895 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,896 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,897 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,898 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,899 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,900 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:37,901 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:38,156 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:38,156 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 for region: 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:38,156 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 84b37956ab3c45570fda115a32fc8299 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:38,199 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:38,205 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/20b4d239aacb53b0c1fb250029e2b764/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:38,206 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/97581af8a5a64d4ea1fc42cf27d9b58d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:53:38,206 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:38,206 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:38,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:38,335 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,335 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:38,335 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:38,335 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:38,336 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:38,337 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:38,337 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:38,337 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:38,337 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:38,338 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:38,339 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-23 10:53:38,339 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:38,339 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:38,340 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:38,340 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:38,340 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:38,341 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:38,341 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,342 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:38,342 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,342 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,342 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,343 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:38,343 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,343 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,343 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:38,344 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,344 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:38,344 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:38,347 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:38,347 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,347 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,347 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,347 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,347 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,347 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-23 10:53:38,348 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:38,349 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:38,349 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788048349 2023-08-23 10:53:38,350 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,350 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:38,353 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:38,354 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:38,354 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:38,354 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788078354 2023-08-23 10:53:38,354 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788138354 2023-08-23 10:53:38,354 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:38,357 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:38,360 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:38,360 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:38,360 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:38,605 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,605 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:38,605 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:38,605 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:38,606 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:38,607 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:38,607 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:38,607 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:38,607 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:38,608 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:38,609 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-23 10:53:38,609 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:38,609 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:38,610 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:38,610 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:38,610 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:38,611 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:38,611 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,612 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:38,612 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,612 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,612 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:38,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,613 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:38,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:38,614 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:38,615 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:38,615 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,615 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,615 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,615 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,615 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,615 DEBUG [Listener at localhost/34585] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,616 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-23 10:53:38,616 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:38,617 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:38,617 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788048617 2023-08-23 10:53:38,618 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:38,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,623 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:38,623 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:38,623 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:38,623 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788078623 2023-08-23 10:53:38,623 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788138623 2023-08-23 10:53:38,623 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:38,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:38,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:38,635 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:38,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:38,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:38,874 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,874 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:38,874 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:38,874 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:38,875 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:38,875 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:38,875 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:38,875 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:38,875 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:38,876 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:38,877 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-23 10:53:38,877 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:38,877 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:38,877 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:38,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:38,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:38,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:38,878 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,879 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:38,879 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,879 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:38,879 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,879 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:38,880 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,880 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:38,880 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:38,881 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,881 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:38,881 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:38,881 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:38,882 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,882 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,882 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,882 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,882 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:38,882 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:38,883 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:38,883 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:38,883 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788048883 2023-08-23 10:53:38,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:38,886 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:38,888 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:38,888 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:38,888 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:38,888 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788078888 2023-08-23 10:53:38,889 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788138889 2023-08-23 10:53:38,889 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:38,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:38,891 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:38,891 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:38,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:38,891 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:39,140 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,140 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:39,140 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:39,140 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:39,141 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:39,141 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:39,141 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:39,141 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:39,141 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:39,142 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:39,143 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-23 10:53:39,143 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:39,143 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:39,143 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:39,143 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,144 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,145 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,145 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:39,145 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,145 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,145 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:39,146 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,146 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:39,146 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:39,147 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:39,147 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,147 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,147 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,147 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,147 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,147 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:39,148 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:39,149 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:39,149 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788049149 2023-08-23 10:53:39,149 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:39,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,153 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:39,153 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:39,153 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:39,153 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788079153 2023-08-23 10:53:39,153 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788139153 2023-08-23 10:53:39,153 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:53:39,154 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:39,156 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:39,156 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:39,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:39,156 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:39,404 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:39,404 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:39,405 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:39,406 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-23 10:53:39,406 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:39,406 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:39,406 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:39,407 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:39,407 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:39,407 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:39,407 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,408 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,409 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:39,409 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,409 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:39,409 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:39,409 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:39,410 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,410 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,410 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,410 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,410 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,410 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:39,410 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:39,411 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:39,411 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788049411 2023-08-23 10:53:39,413 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:39,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,415 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:39,415 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:39,415 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:39,415 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788079415 2023-08-23 10:53:39,415 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788139415 2023-08-23 10:53:39,415 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:53:39,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:39,419 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN] 2023-08-23 10:53:39,419 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:39,419 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:39,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:39,419 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:39,666 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,666 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:39,666 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:39,666 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:39,667 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:39,667 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:39,667 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:39,667 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:39,667 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:39,668 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:39,669 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-23 10:53:39,669 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:53:39,669 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:53:39,669 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:39,669 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:39,670 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,671 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:39,672 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:39,673 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,673 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,673 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,673 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,673 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:39,674 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:39,674 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:39,674 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:39,675 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788049675 2023-08-23 10:53:39,676 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:39,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,678 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:39,679 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:39,680 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:39,680 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:39,680 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788079680 2023-08-23 10:53:39,680 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788139680 2023-08-23 10:53:39,680 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:39,680 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:39,685 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:39,685 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:39,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:39,685 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:39,930 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,930 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:39,930 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:39,930 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:39,933 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:39,933 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:39,933 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:39,933 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:39,933 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:39,934 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:39,934 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:53:39,935 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:39,935 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:39,935 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:39,935 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:39,935 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:39,936 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,936 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:39,936 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,936 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:39,936 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:39,937 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:39,938 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,938 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,938 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,938 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:39,938 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:39,938 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:39,939 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:39,939 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:39,939 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788049939 2023-08-23 10:53:39,940 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:39,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:39,943 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:39,945 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:39,945 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:39,945 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:39,945 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788079945 2023-08-23 10:53:39,945 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788139945 2023-08-23 10:53:39,945 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:39,945 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:39,948 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:39,948 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:39,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:39,948 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:40,196 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:40,196 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:40,197 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:40,198 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-23 10:53:40,198 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:40,198 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:40,199 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:40,199 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:40,199 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:40,199 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,199 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,200 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,201 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:40,201 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,201 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,201 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:40,201 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,201 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,201 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,201 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:40,201 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-08-23 10:53:40,202 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:40,202 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:40,203 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:40,203 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788050203 2023-08-23 10:53:40,203 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:40,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,206 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,208 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:40,208 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:40,208 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:40,208 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788080208 2023-08-23 10:53:40,208 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788140208 2023-08-23 10:53:40,208 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:40,208 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:40,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:40,211 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-08-23 10:53:40,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:40,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:40,459 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:40,459 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:40,460 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:40,461 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-23 10:53:40,461 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-08-23 10:53:40,461 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:40,462 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:40,462 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:40,462 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:40,462 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,462 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:40,463 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,463 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,463 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,463 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:40,464 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,464 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,464 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:40,465 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,465 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,465 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:40,465 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,465 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,465 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,465 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:53:40,466 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-08-23 10:53:40,466 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:40,467 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:40,467 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:40,468 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788050468 2023-08-23 10:53:40,469 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:40,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,474 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:40,474 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:40,474 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:40,474 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788080474 2023-08-23 10:53:40,474 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788140474 2023-08-23 10:53:40,474 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:53:40,474 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:53:40,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d exec-time=5.2610 sec 2023-08-23 10:53:40,477 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,478 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=true 2023-08-23 10:53:40,628 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:40,629 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:40,629 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788020629"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788020629"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788020629"}]},"ts":"1692788020629"} 2023-08-23 10:53:40,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:40,631 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:53:40,631 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:53:40,631 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:53:40,724 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:40,725 WARN [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:53:40,725 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:53:40,725 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:53:40,726 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:53:40,727 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-08-23 10:53:40,727 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:53:40,727 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:53:40,727 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:53:40,727 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:53:40,728 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,728 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:53:40,728 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,728 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:53:40,728 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:53:40,729 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,730 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:53:40,730 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,730 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-08-23 10:53:40,730 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:53:40,731 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:53:40,731 INFO [Listener at localhost/34585] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299 to restore RIT 2023-08-23 10:53:40,736 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788050736 2023-08-23 10:53:40,737 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:53:40,737 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:53:40,739 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:53:40,740 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=84b37956ab3c45570fda115a32fc8299, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:53:40,740 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:53:40,740 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788080740 2023-08-23 10:53:40,740 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788140740 2023-08-23 10:53:40,740 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:53:40,740 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:40,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788020740"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788020740"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788020740"}]},"ts":"1692788020740"} 2023-08-23 10:53:40,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:40,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:40,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 84b37956ab3c45570fda115a32fc8299, NAME => 'testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:40,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:40,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:40,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,901 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,902 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1 2023-08-23 10:53:40,902 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1 2023-08-23 10:53:40,902 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b37956ab3c45570fda115a32fc8299 columnFamilyName cf1 2023-08-23 10:53:40,910 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 2023-08-23 10:53:40,913 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf 2023-08-23 10:53:40,917 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf 2023-08-23 10:53:40,920 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc 2023-08-23 10:53:40,924 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 2023-08-23 10:53:40,929 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 2023-08-23 10:53:40,933 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 2023-08-23 10:53:40,934 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(310): Store=84b37956ab3c45570fda115a32fc8299/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:40,934 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,934 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2 2023-08-23 10:53:40,935 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2 2023-08-23 10:53:40,935 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b37956ab3c45570fda115a32fc8299 columnFamilyName cf2 2023-08-23 10:53:40,945 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b 2023-08-23 10:53:40,950 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 2023-08-23 10:53:40,954 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 2023-08-23 10:53:40,957 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c 2023-08-23 10:53:40,960 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 2023-08-23 10:53:40,963 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d 2023-08-23 10:53:40,967 DEBUG [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 2023-08-23 10:53:40,967 INFO [StoreOpener-84b37956ab3c45570fda115a32fc8299-1] regionserver.HStore(310): Store=84b37956ab3c45570fda115a32fc8299/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:40,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.splits 2023-08-23 10:53:40,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:40,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:40,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 84b37956ab3c45570fda115a32fc8299; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63303712, jitterRate=-0.0567011833190918}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:40,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:40,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299., pid=48, masterSystemTime=1692788020895 2023-08-23 10:53:40,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:40,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:40,975 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:40,975 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:40,976 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-23 10:53:40,976 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-23 10:53:40,977 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): 84b37956ab3c45570fda115a32fc8299/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:40,977 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): 84b37956ab3c45570fda115a32fc8299/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:40,977 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 84b37956ab3c45570fda115a32fc8299/cf1 in testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:40,977 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 84b37956ab3c45570fda115a32fc8299/cf2 in testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:40,978 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp, totalSize=34.9 K 2023-08-23 10:53:40,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:40,978 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp, totalSize=34.9 K 2023-08-23 10:53:40,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:40,978 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting e841c3bbc9f84632ab0fd6a8f2692e27, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788013743 2023-08-23 10:53:40,978 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:40,978 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting c427a71642ba43aca96fd823cb32b86d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788013743 2023-08-23 10:53:40,978 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788020978"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788020978"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788020978"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788020978"}]},"ts":"1692788020978"} 2023-08-23 10:53:40,978 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 36a6c20ebe1a411592a349d8bc0b4bdf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788013946 2023-08-23 10:53:40,979 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 32d518f51c0a4c7ba83cf81a44b39887, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788013946 2023-08-23 10:53:40,979 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting eeeef2c2015947668d988558e7e6bcf6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788014153 2023-08-23 10:53:40,979 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting ae8abafb56d942208ab2fd40791a960c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788014153 2023-08-23 10:53:40,979 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 6e52ed8c4f674ff9aced32139482eff3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788014351 2023-08-23 10:53:40,980 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting ba45ff54ef0241cd93bb9d9ee3d5f949, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788014351 2023-08-23 10:53:40,980 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 427e8da914bd476390bef84aaa95bcaf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788014568 2023-08-23 10:53:40,980 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting e6343a0cff6d415aa5d9d7ba35ab5a04, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788014568 2023-08-23 10:53:40,980 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 095a08d0d6f04fdc979670d18d5b3100, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788014757 2023-08-23 10:53:40,980 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 7eda1bd8d1ba446f994fca20b05e7ca9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788014757 2023-08-23 10:53:40,981 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 49a90f1d1d024f6b8fc65ac6c2726dcc, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788014952 2023-08-23 10:53:40,981 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 2b0a01d941fe41448c0b22aa55d9065b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788014952 2023-08-23 10:53:40,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-08-23 10:53:40,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 in 239 msec 2023-08-23 10:53:40,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN in 1.5650 sec 2023-08-23 10:53:40,991 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-08-23 10:53:40,994 DEBUG [Listener at localhost/34585] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-08-23 10:53:40,996 INFO [Listener at localhost/34585] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-23 10:53:40,999 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-08-23 10:53:41,000 INFO [Listener at localhost/34585] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-08-23 10:53:41,000 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-08-23 10:53:41,018 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): 84b37956ab3c45570fda115a32fc8299#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:41,020 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): 84b37956ab3c45570fda115a32fc8299#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:41,046 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf1/8f5f95f63e594c8bb65c508b9606cf0b as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/8f5f95f63e594c8bb65c508b9606cf0b 2023-08-23 10:53:41,047 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/.tmp/cf2/afd872c6d836451abf3348e73baf314e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/afd872c6d836451abf3348e73baf314e 2023-08-23 10:53:41,055 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 84b37956ab3c45570fda115a32fc8299/cf2 of 84b37956ab3c45570fda115a32fc8299 into afd872c6d836451abf3348e73baf314e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:41,055 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:41,055 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299., storeName=84b37956ab3c45570fda115a32fc8299/cf2, priority=9, startTime=1692788020974; duration=0sec 2023-08-23 10:53:41,056 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:41,056 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 84b37956ab3c45570fda115a32fc8299/cf1 of 84b37956ab3c45570fda115a32fc8299 into 8f5f95f63e594c8bb65c508b9606cf0b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:41,056 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:41,057 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299., storeName=84b37956ab3c45570fda115a32fc8299/cf1, priority=9, startTime=1692788020974; duration=0sec 2023-08-23 10:53:41,057 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:41,071 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:41,073 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-08-23 10:53:41,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-08-23 10:53:41,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-23 10:53:41,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788021077"}]},"ts":"1692788021077"} 2023-08-23 10:53:41,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-23 10:53:41,080 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-08-23 10:53:41,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN}] 2023-08-23 10:53:41,082 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN 2023-08-23 10:53:41,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:41,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788021082"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788021082"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788021082"}]},"ts":"1692788021082"} 2023-08-23 10:53:41,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:41,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-23 10:53:41,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:41,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 84b37956ab3c45570fda115a32fc8299, disabling compactions & flushes 2023-08-23 10:53:41,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:41,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:41,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. after waiting 0 ms 2023-08-23 10:53:41,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:41,237 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc] to archive 2023-08-23 10:53:41,238 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:41,240 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/e841c3bbc9f84632ab0fd6a8f2692e27 2023-08-23 10:53:41,241 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/36a6c20ebe1a411592a349d8bc0b4bdf 2023-08-23 10:53:41,242 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/eeeef2c2015947668d988558e7e6bcf6 2023-08-23 10:53:41,243 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/6e52ed8c4f674ff9aced32139482eff3 2023-08-23 10:53:41,245 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/427e8da914bd476390bef84aaa95bcaf 2023-08-23 10:53:41,246 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/095a08d0d6f04fdc979670d18d5b3100 2023-08-23 10:53:41,248 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/49a90f1d1d024f6b8fc65ac6c2726dcc 2023-08-23 10:53:41,253 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b] to archive 2023-08-23 10:53:41,254 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:41,256 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/c427a71642ba43aca96fd823cb32b86d 2023-08-23 10:53:41,257 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/32d518f51c0a4c7ba83cf81a44b39887 2023-08-23 10:53:41,258 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ae8abafb56d942208ab2fd40791a960c 2023-08-23 10:53:41,260 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/ba45ff54ef0241cd93bb9d9ee3d5f949 2023-08-23 10:53:41,261 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/e6343a0cff6d415aa5d9d7ba35ab5a04 2023-08-23 10:53:41,262 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/7eda1bd8d1ba446f994fca20b05e7ca9 2023-08-23 10:53:41,263 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/2b0a01d941fe41448c0b22aa55d9065b 2023-08-23 10:53:41,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:53:41,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:41,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299. 2023-08-23 10:53:41,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 84b37956ab3c45570fda115a32fc8299: 2023-08-23 10:53:41,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:41,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=84b37956ab3c45570fda115a32fc8299, regionState=CLOSED 2023-08-23 10:53:41,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788021270"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788021270"}]},"ts":"1692788021270"} 2023-08-23 10:53:41,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-08-23 10:53:41,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 84b37956ab3c45570fda115a32fc8299, server=jenkins-hbase4.apache.org,43499,1692787997532 in 187 msec 2023-08-23 10:53:41,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-08-23 10:53:41,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, UNASSIGN in 192 msec 2023-08-23 10:53:41,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788021275"}]},"ts":"1692788021275"} 2023-08-23 10:53:41,276 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-23 10:53:41,278 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-08-23 10:53:41,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 205 msec 2023-08-23 10:53:41,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-08-23 10:53:41,380 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-08-23 10:53:41,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-08-23 10:53:41,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,383 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-23 10:53:41,387 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:41,389 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits] 2023-08-23 10:53:41,392 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/8f5f95f63e594c8bb65c508b9606cf0b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf1/8f5f95f63e594c8bb65c508b9606cf0b 2023-08-23 10:53:41,394 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/afd872c6d836451abf3348e73baf314e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/cf2/afd872c6d836451abf3348e73baf314e 2023-08-23 10:53:41,396 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299/recovered.edits/90.seqid 2023-08-23 10:53:41,397 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackAndDoubleExecution/84b37956ab3c45570fda115a32fc8299 2023-08-23 10:53:41,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-08-23 10:53:41,400 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,402 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-08-23 10:53:41,404 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-08-23 10:53:41,405 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,405 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-08-23 10:53:41,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788021405"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:41,406 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-23 10:53:41,407 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 84b37956ab3c45570fda115a32fc8299, NAME => 'testRollbackAndDoubleExecution,,1692788013233.84b37956ab3c45570fda115a32fc8299.', STARTKEY => '', ENDKEY => ''}] 2023-08-23 10:53:41,407 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-08-23 10:53:41,407 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788021407"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:41,408 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-08-23 10:53:41,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:53:41,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-08-23 10:53:41,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-08-23 10:53:41,486 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-08-23 10:53:41,500 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=406 (was 407), OpenFileDescriptor=870 (was 857) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 337), ProcessCount=171 (was 171), AvailableMemoryMB=10856 (was 10974) 2023-08-23 10:53:41,514 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406, OpenFileDescriptor=870, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=171, AvailableMemoryMB=10855 2023-08-23 10:53:41,514 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:41,517 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:41,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:41,519 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:41,520 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:41,521 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:41,523 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,523 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 empty. 2023-08-23 10:53:41,524 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,524 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-23 10:53:41,541 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:41,542 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1a4eca4d5307f5febb75fe13c4379735, NAME => 'testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:41,555 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:41,555 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 1a4eca4d5307f5febb75fe13c4379735, disabling compactions & flushes 2023-08-23 10:53:41,555 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,555 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,555 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. after waiting 0 ms 2023-08-23 10:53:41,555 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,555 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,556 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 1a4eca4d5307f5febb75fe13c4379735: 2023-08-23 10:53:41,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:41,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788021559"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788021559"}]},"ts":"1692788021559"} 2023-08-23 10:53:41,561 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:41,562 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:41,562 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788021562"}]},"ts":"1692788021562"} 2023-08-23 10:53:41,563 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-08-23 10:53:41,566 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:41,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:41,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:41,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:41,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:41,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, ASSIGN}] 2023-08-23 10:53:41,569 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, ASSIGN 2023-08-23 10:53:41,570 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:53:41,720 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:41,721 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=1a4eca4d5307f5febb75fe13c4379735, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:41,722 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788021721"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788021721"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788021721"}]},"ts":"1692788021721"} 2023-08-23 10:53:41,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 1a4eca4d5307f5febb75fe13c4379735, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:41,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1a4eca4d5307f5febb75fe13c4379735, NAME => 'testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:41,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:41,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:41,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,880 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,881 DEBUG [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf1 2023-08-23 10:53:41,881 DEBUG [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf1 2023-08-23 10:53:41,882 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a4eca4d5307f5febb75fe13c4379735 columnFamilyName cf1 2023-08-23 10:53:41,882 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] regionserver.HStore(310): Store=1a4eca4d5307f5febb75fe13c4379735/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:41,882 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,884 DEBUG [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf2 2023-08-23 10:53:41,884 DEBUG [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf2 2023-08-23 10:53:41,885 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1a4eca4d5307f5febb75fe13c4379735 columnFamilyName cf2 2023-08-23 10:53:41,885 INFO [StoreOpener-1a4eca4d5307f5febb75fe13c4379735-1] regionserver.HStore(310): Store=1a4eca4d5307f5febb75fe13c4379735/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:41,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:41,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:41,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:41,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1a4eca4d5307f5febb75fe13c4379735; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62520977, jitterRate=-0.06836484372615814}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:41,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1a4eca4d5307f5febb75fe13c4379735: 2023-08-23 10:53:41,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735., pid=55, masterSystemTime=1692788021875 2023-08-23 10:53:41,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:41,897 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=1a4eca4d5307f5febb75fe13c4379735, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:41,897 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788021897"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788021897"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788021897"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788021897"}]},"ts":"1692788021897"} 2023-08-23 10:53:41,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-08-23 10:53:41,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 1a4eca4d5307f5febb75fe13c4379735, server=jenkins-hbase4.apache.org,43499,1692787997532 in 175 msec 2023-08-23 10:53:41,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-08-23 10:53:41,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, ASSIGN in 333 msec 2023-08-23 10:53:41,902 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:41,903 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788021903"}]},"ts":"1692788021903"} 2023-08-23 10:53:41,904 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-08-23 10:53:41,906 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:41,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 388 msec 2023-08-23 10:53:42,020 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,022 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,023 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,023 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, UNASSIGN}] 2023-08-23 10:53:42,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, UNASSIGN 2023-08-23 10:53:42,026 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=1a4eca4d5307f5febb75fe13c4379735, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788022026"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788022026"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788022026"}]},"ts":"1692788022026"} 2023-08-23 10:53:42,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 1a4eca4d5307f5febb75fe13c4379735, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:42,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:42,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1a4eca4d5307f5febb75fe13c4379735, disabling compactions & flushes 2023-08-23 10:53:42,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:42,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:42,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. after waiting 0 ms 2023-08-23 10:53:42,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-23 10:53:42,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:42,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735. 2023-08-23 10:53:42,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1a4eca4d5307f5febb75fe13c4379735: 2023-08-23 10:53:42,187 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:42,188 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=1a4eca4d5307f5febb75fe13c4379735, regionState=CLOSED 2023-08-23 10:53:42,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788022188"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788022188"}]},"ts":"1692788022188"} 2023-08-23 10:53:42,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-08-23 10:53:42,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 1a4eca4d5307f5febb75fe13c4379735, server=jenkins-hbase4.apache.org,43499,1692787997532 in 161 msec 2023-08-23 10:53:42,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-08-23 10:53:42,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=1a4eca4d5307f5febb75fe13c4379735, UNASSIGN in 168 msec 2023-08-23 10:53:42,220 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-23 10:53:42,222 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-08-23 10:53:42,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1692788022223"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1692788022223"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1692788022223"}]},"ts":"1692788022223"} 2023-08-23 10:53:42,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022223"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788022223"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788022223"}]},"ts":"1692788022223"} 2023-08-23 10:53:42,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022223"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788022223"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788022223"}]},"ts":"1692788022223"} 2023-08-23 10:53:42,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, ASSIGN}] 2023-08-23 10:53:42,227 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, ASSIGN 2023-08-23 10:53:42,228 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:53:42,228 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, ASSIGN 2023-08-23 10:53:42,229 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:53:42,378 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:53:42,380 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=ffedba6c94ad43305ebb2566148be64d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,380 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022379"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788022379"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788022379"}]},"ts":"1692788022379"} 2023-08-23 10:53:42,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure ffedba6c94ad43305ebb2566148be64d, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:42,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=88c3108086fd6ceee8569a37c2048096, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,384 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022384"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788022384"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788022384"}]},"ts":"1692788022384"} 2023-08-23 10:53:42,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 88c3108086fd6ceee8569a37c2048096, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:42,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88c3108086fd6ceee8569a37c2048096, NAME => 'testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.', STARTKEY => '41', ENDKEY => ''} 2023-08-23 10:53:42,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:42,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:42,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,538 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,540 DEBUG [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf1 2023-08-23 10:53:42,540 DEBUG [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf1 2023-08-23 10:53:42,540 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88c3108086fd6ceee8569a37c2048096 columnFamilyName cf1 2023-08-23 10:53:42,541 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] regionserver.HStore(310): Store=88c3108086fd6ceee8569a37c2048096/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:42,541 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,542 DEBUG [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf2 2023-08-23 10:53:42,542 DEBUG [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf2 2023-08-23 10:53:42,543 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88c3108086fd6ceee8569a37c2048096 columnFamilyName cf2 2023-08-23 10:53:42,543 INFO [StoreOpener-88c3108086fd6ceee8569a37c2048096-1] regionserver.HStore(310): Store=88c3108086fd6ceee8569a37c2048096/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:42,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:42,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88c3108086fd6ceee8569a37c2048096; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73588522, jitterRate=0.09655442833900452}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:42,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88c3108086fd6ceee8569a37c2048096: 2023-08-23 10:53:42,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096., pid=62, masterSystemTime=1692788022533 2023-08-23 10:53:42,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ffedba6c94ad43305ebb2566148be64d, NAME => 'testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.', STARTKEY => '', ENDKEY => '41'} 2023-08-23 10:53:42,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=88c3108086fd6ceee8569a37c2048096, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022550"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788022550"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788022550"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788022550"}]},"ts":"1692788022550"} 2023-08-23 10:53:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,552 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,553 DEBUG [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf1 2023-08-23 10:53:42,553 DEBUG [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf1 2023-08-23 10:53:42,554 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffedba6c94ad43305ebb2566148be64d columnFamilyName cf1 2023-08-23 10:53:42,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-08-23 10:53:42,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 88c3108086fd6ceee8569a37c2048096, server=jenkins-hbase4.apache.org,43499,1692787997532 in 166 msec 2023-08-23 10:53:42,554 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] regionserver.HStore(310): Store=ffedba6c94ad43305ebb2566148be64d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:42,554 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,556 DEBUG [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf2 2023-08-23 10:53:42,556 DEBUG [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf2 2023-08-23 10:53:42,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, ASSIGN in 328 msec 2023-08-23 10:53:42,556 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffedba6c94ad43305ebb2566148be64d columnFamilyName cf2 2023-08-23 10:53:42,557 INFO [StoreOpener-ffedba6c94ad43305ebb2566148be64d-1] regionserver.HStore(310): Store=ffedba6c94ad43305ebb2566148be64d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:42,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:42,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ffedba6c94ad43305ebb2566148be64d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63055033, jitterRate=-0.06040678918361664}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:42,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ffedba6c94ad43305ebb2566148be64d: 2023-08-23 10:53:42,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d., pid=61, masterSystemTime=1692788022533 2023-08-23 10:53:42,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,564 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=ffedba6c94ad43305ebb2566148be64d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022564"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788022564"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788022564"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788022564"}]},"ts":"1692788022564"} 2023-08-23 10:53:42,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-08-23 10:53:42,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure ffedba6c94ad43305ebb2566148be64d, server=jenkins-hbase4.apache.org,43499,1692787997532 in 184 msec 2023-08-23 10:53:42,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-08-23 10:53:42,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, ASSIGN in 341 msec 2023-08-23 10:53:42,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 in 549 msec 2023-08-23 10:53:42,776 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:53:42,779 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:42,780 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-08-23 10:53:42,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-08-23 10:53:42,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:42,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-23 10:53:42,784 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788022784"}]},"ts":"1692788022784"} 2023-08-23 10:53:42,785 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-08-23 10:53:42,788 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-08-23 10:53:42,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, UNASSIGN}] 2023-08-23 10:53:42,789 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, UNASSIGN 2023-08-23 10:53:42,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=88c3108086fd6ceee8569a37c2048096, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022790"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788022790"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788022790"}]},"ts":"1692788022790"} 2023-08-23 10:53:42,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 88c3108086fd6ceee8569a37c2048096, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:42,793 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, UNASSIGN 2023-08-23 10:53:42,794 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=ffedba6c94ad43305ebb2566148be64d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:42,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022794"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788022794"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788022794"}]},"ts":"1692788022794"} 2023-08-23 10:53:42,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure ffedba6c94ad43305ebb2566148be64d, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:42,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-23 10:53:42,943 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ffedba6c94ad43305ebb2566148be64d, disabling compactions & flushes 2023-08-23 10:53:42,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. after waiting 0 ms 2023-08-23 10:53:42,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-23 10:53:42,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:42,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d. 2023-08-23 10:53:42,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ffedba6c94ad43305ebb2566148be64d: 2023-08-23 10:53:42,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:42,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88c3108086fd6ceee8569a37c2048096, disabling compactions & flushes 2023-08-23 10:53:42,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. after waiting 0 ms 2023-08-23 10:53:42,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,952 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=ffedba6c94ad43305ebb2566148be64d, regionState=CLOSED 2023-08-23 10:53:42,952 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022952"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788022952"}]},"ts":"1692788022952"} 2023-08-23 10:53:42,955 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-23 10:53:42,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-08-23 10:53:42,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure ffedba6c94ad43305ebb2566148be64d, server=jenkins-hbase4.apache.org,43499,1692787997532 in 158 msec 2023-08-23 10:53:42,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:42,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096. 2023-08-23 10:53:42,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88c3108086fd6ceee8569a37c2048096: 2023-08-23 10:53:42,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ffedba6c94ad43305ebb2566148be64d, UNASSIGN in 167 msec 2023-08-23 10:53:42,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:42,957 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=88c3108086fd6ceee8569a37c2048096, regionState=CLOSED 2023-08-23 10:53:42,957 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788022957"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788022957"}]},"ts":"1692788022957"} 2023-08-23 10:53:42,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-08-23 10:53:42,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 88c3108086fd6ceee8569a37c2048096, server=jenkins-hbase4.apache.org,43499,1692787997532 in 167 msec 2023-08-23 10:53:42,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-08-23 10:53:42,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=88c3108086fd6ceee8569a37c2048096, UNASSIGN in 172 msec 2023-08-23 10:53:42,962 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788022962"}]},"ts":"1692788022962"} 2023-08-23 10:53:42,963 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-08-23 10:53:42,965 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-08-23 10:53:42,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-08-23 10:53:43,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-08-23 10:53:43,087 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-08-23 10:53:43,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,090 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,091 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-23 10:53:43,096 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:43,096 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:43,096 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:43,098 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/recovered.edits] 2023-08-23 10:53:43,098 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/recovered.edits] 2023-08-23 10:53:43,098 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/recovered.edits] 2023-08-23 10:53:43,104 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/recovered.edits/4.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735/recovered.edits/4.seqid 2023-08-23 10:53:43,104 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/recovered.edits/7.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096/recovered.edits/7.seqid 2023-08-23 10:53:43,104 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/recovered.edits/7.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d/recovered.edits/7.seqid 2023-08-23 10:53:43,105 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/1a4eca4d5307f5febb75fe13c4379735 2023-08-23 10:53:43,105 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/88c3108086fd6ceee8569a37c2048096 2023-08-23 10:53:43,105 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionNoStoreFile/ffedba6c94ad43305ebb2566148be64d 2023-08-23 10:53:43,105 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-08-23 10:53:43,111 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,113 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-08-23 10:53:43,114 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-08-23 10:53:43,115 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,115 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-08-23 10:53:43,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788023115"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:43,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788023115"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:43,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788023115"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:43,117 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:53:43,117 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1a4eca4d5307f5febb75fe13c4379735, NAME => 'testSplitTableRegionNoStoreFile,,1692788021518.1a4eca4d5307f5febb75fe13c4379735.', STARTKEY => '', ENDKEY => ''}, {ENCODED => ffedba6c94ad43305ebb2566148be64d, NAME => 'testSplitTableRegionNoStoreFile,,1692788022020.ffedba6c94ad43305ebb2566148be64d.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 88c3108086fd6ceee8569a37c2048096, NAME => 'testSplitTableRegionNoStoreFile,41,1692788022020.88c3108086fd6ceee8569a37c2048096.', STARTKEY => '41', ENDKEY => ''}] 2023-08-23 10:53:43,117 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-08-23 10:53:43,117 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788023117"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:43,119 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-08-23 10:53:43,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:53:43,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 33 msec 2023-08-23 10:53:43,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-08-23 10:53:43,193 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-08-23 10:53:43,207 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=409 (was 406) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-712250612_17 at /127.0.0.1:58894 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-712250612_17 at /127.0.0.1:54290 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=868 (was 870), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 300), ProcessCount=171 (was 171), AvailableMemoryMB=10855 (was 10855) 2023-08-23 10:53:43,222 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=409, OpenFileDescriptor=868, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=171, AvailableMemoryMB=10854 2023-08-23 10:53:43,222 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:43,224 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:43,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:43,226 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,227 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:43,228 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:43,230 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,230 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f empty. 2023-08-23 10:53:43,231 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-23 10:53:43,248 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:43,249 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8cd300f31326ee729e983413d0ee732f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 8cd300f31326ee729e983413d0ee732f, disabling compactions & flushes 2023-08-23 10:53:43,262 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. after waiting 0 ms 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,262 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,262 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:43,265 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:43,266 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788023266"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788023266"}]},"ts":"1692788023266"} 2023-08-23 10:53:43,267 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:43,268 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:43,268 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788023268"}]},"ts":"1692788023268"} 2023-08-23 10:53:43,270 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-08-23 10:53:43,273 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:43,275 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:43,275 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:43,275 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:43,275 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:43,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, ASSIGN}] 2023-08-23 10:53:43,276 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, ASSIGN 2023-08-23 10:53:43,277 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:43,427 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:43,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8cd300f31326ee729e983413d0ee732f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788023429"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788023429"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788023429"}]},"ts":"1692788023429"} 2023-08-23 10:53:43,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 8cd300f31326ee729e983413d0ee732f, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:43,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8cd300f31326ee729e983413d0ee732f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:43,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:43,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:43,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,590 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,592 DEBUG [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1 2023-08-23 10:53:43,592 DEBUG [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1 2023-08-23 10:53:43,592 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8cd300f31326ee729e983413d0ee732f columnFamilyName cf1 2023-08-23 10:53:43,593 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] regionserver.HStore(310): Store=8cd300f31326ee729e983413d0ee732f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:43,593 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,594 DEBUG [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2 2023-08-23 10:53:43,595 DEBUG [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2 2023-08-23 10:53:43,595 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8cd300f31326ee729e983413d0ee732f columnFamilyName cf2 2023-08-23 10:53:43,595 INFO [StoreOpener-8cd300f31326ee729e983413d0ee732f-1] regionserver.HStore(310): Store=8cd300f31326ee729e983413d0ee732f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:43,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:43,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:43,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:43,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8cd300f31326ee729e983413d0ee732f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74440953, jitterRate=0.10925664007663727}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:43,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:43,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f., pid=71, masterSystemTime=1692788023582 2023-08-23 10:53:43,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=8cd300f31326ee729e983413d0ee732f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788023606"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788023606"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788023606"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788023606"}]},"ts":"1692788023606"} 2023-08-23 10:53:43,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-08-23 10:53:43,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 8cd300f31326ee729e983413d0ee732f, server=jenkins-hbase4.apache.org,41821,1692787997489 in 178 msec 2023-08-23 10:53:43,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-08-23 10:53:43,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, ASSIGN in 335 msec 2023-08-23 10:53:43,612 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:43,612 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788023612"}]},"ts":"1692788023612"} 2023-08-23 10:53:43,614 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-08-23 10:53:43,616 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:43,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 391 msec 2023-08-23 10:53:43,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:43,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,736 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:43,736 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,736 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:43,736 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:43,737 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,737 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,738 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,738 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,738 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,738 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,739 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,739 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:43,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:43,741 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,741 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:43,742 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:53:43,744 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,745 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,745 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,745 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,746 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,746 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,754 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/0f1f408d6ff540569144304835bc4fd7 2023-08-23 10:53:43,772 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/1b424ec7a42047f4889c49c3b7a7ed98 2023-08-23 10:53:43,776 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/0f1f408d6ff540569144304835bc4fd7 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7 2023-08-23 10:53:43,780 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:43,781 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/1b424ec7a42047f4889c49c3b7a7ed98 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98 2023-08-23 10:53:43,785 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:53:43,786 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 44ms, sequenceid=6, compaction requested=false 2023-08-23 10:53:43,786 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:43,786 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:43,786 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,788 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e53dcda[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:43,790 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:43,790 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,792 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,792 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:43,792 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:43,799 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,799 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,799 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,800 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,800 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:43,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:43,803 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:43,803 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:43,803 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:43,804 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,804 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,805 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,806 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,806 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,807 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:43,808 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,808 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,809 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:43,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,817 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,817 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:43,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:43,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,818 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,918 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:43,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:43,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,934 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:43,934 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,934 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:43,934 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:43,935 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,935 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,936 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,936 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,936 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,936 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,937 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:43,938 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,939 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,939 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:43,939 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,939 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,938 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:43,939 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,948 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/a4071ad9fc9b49c998227632a7831a72 2023-08-23 10:53:43,961 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/ccc5695eba4d46f3b9578112b9b6c175 2023-08-23 10:53:43,966 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/a4071ad9fc9b49c998227632a7831a72 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72 2023-08-23 10:53:43,970 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:43,970 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/ccc5695eba4d46f3b9578112b9b6c175 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175 2023-08-23 10:53:43,974 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:53:43,975 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 36ms, sequenceid=19, compaction requested=false 2023-08-23 10:53:43,975 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:43,975 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:43,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59708b32[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:43,979 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:43,979 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,985 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,985 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:43,985 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,987 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:43,988 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:43,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:43,991 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:43,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:43,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:43,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,992 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:43,996 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:43,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,997 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:43,997 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:43,998 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:43,999 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,000 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,000 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,003 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,003 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,004 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,104 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:44,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:44,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:44,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:44,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,119 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:44,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:44,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,123 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,123 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,123 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,123 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:44,124 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,125 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,125 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,126 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,126 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,126 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,126 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,133 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/fe55f2095c024cdfadb9d8c784fb5f24 2023-08-23 10:53:44,145 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 2023-08-23 10:53:44,150 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/fe55f2095c024cdfadb9d8c784fb5f24 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24 2023-08-23 10:53:44,154 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:44,154 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 2023-08-23 10:53:44,158 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:53:44,159 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 35ms, sequenceid=32, compaction requested=true 2023-08-23 10:53:44,159 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:44,159 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:44,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:44,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,164 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,165 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,165 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:44,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aa2b987[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:44,167 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:44,167 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,168 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,168 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,169 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,169 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:44,169 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,170 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,171 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,171 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:44,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:44,174 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:44,175 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:44,175 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:44,175 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,175 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,177 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,178 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,179 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,179 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,184 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,184 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,185 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,285 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:44,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:44,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:44,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:44,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,300 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:44,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:44,301 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,301 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,302 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,302 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,303 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,303 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,303 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,304 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,304 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:44,305 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,305 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:44,305 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:44,306 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,307 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,307 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,307 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,307 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,307 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,315 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 2023-08-23 10:53:44,331 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/a8036d1b54ea4101a1e8cd70d83768ee 2023-08-23 10:53:44,336 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 2023-08-23 10:53:44,340 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:44,341 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/a8036d1b54ea4101a1e8cd70d83768ee as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee 2023-08-23 10:53:44,345 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:53:44,346 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 41ms, sequenceid=45, compaction requested=true 2023-08-23 10:53:44,346 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:44,346 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:44,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:44,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,348 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,348 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,348 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,348 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@661d4ca9[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:44,350 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:44,350 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:44,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,352 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,352 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,352 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,353 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,359 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,359 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,359 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:44,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:44,362 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:44,362 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:44,362 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:44,362 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,362 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,367 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,367 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,368 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,369 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,369 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,370 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,372 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,373 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,373 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,373 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,373 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,373 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,373 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,374 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,374 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,374 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,374 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,374 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,473 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:44,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:44,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:44,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:44,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,487 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,487 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,488 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:44,488 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:44,488 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,488 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,490 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,490 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,490 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,490 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:44,493 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:44,493 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,493 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:44,493 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:44,495 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,508 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/4dfe91412d1f4e6fa9832019f87732c7 2023-08-23 10:53:44,528 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/857e7eb8324d4574a7855440d98b7c4e 2023-08-23 10:53:44,533 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/4dfe91412d1f4e6fa9832019f87732c7 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7 2023-08-23 10:53:44,537 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:44,538 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/857e7eb8324d4574a7855440d98b7c4e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e 2023-08-23 10:53:44,542 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:53:44,543 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 50ms, sequenceid=58, compaction requested=true 2023-08-23 10:53:44,543 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:44,543 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,543 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:44,543 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:44,543 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,543 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,543 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,545 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,545 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,546 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,546 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:44,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16d8e9bc[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:44,548 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:44,548 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,551 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:44,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,554 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:44,558 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:44,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:44,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:44,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,558 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,560 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,561 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,561 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,562 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,569 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,569 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,568 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,569 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,569 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,569 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,570 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,670 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:44,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:44,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:44,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:44,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,685 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:44,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:44,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,690 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,690 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,690 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,690 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:44,690 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,692 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:44,692 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,692 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:44,692 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:44,692 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,694 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,701 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/0a934819e8dd43d7b50d648d80e3f374 2023-08-23 10:53:44,714 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/5db80d0ecff54b40b72a6e42d338be1d 2023-08-23 10:53:44,718 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/0a934819e8dd43d7b50d648d80e3f374 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374 2023-08-23 10:53:44,722 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:44,723 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/5db80d0ecff54b40b72a6e42d338be1d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d 2023-08-23 10:53:44,726 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:53:44,727 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 35ms, sequenceid=71, compaction requested=true 2023-08-23 10:53:44,727 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:44,727 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,730 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,730 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,731 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,731 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bacc94e[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:44,733 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:44,733 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:44,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,734 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:44,736 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,736 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,736 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,737 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,737 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,737 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,737 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,738 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:44,742 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:44,742 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:44,742 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:44,742 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,742 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,744 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,745 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,746 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,746 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,748 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,748 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,749 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,753 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,752 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,753 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,753 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,753 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,853 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:44,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:44,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:44,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:44,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,868 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:44,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:44,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,870 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,870 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,870 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,871 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:44,871 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,872 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,872 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:44,872 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-08-23 10:53:44,875 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,875 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,875 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,875 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,876 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,876 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,883 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:44,887 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:44,895 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:44,898 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:44,899 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/9383c22c8ee8467a8b5af802cb00a47c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:44,903 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:44,903 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c, entries=9, sequenceid=83, filesize=5.1 K 2023-08-23 10:53:44,903 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:44,907 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:44,907 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597, entries=9, sequenceid=83, filesize=5.1 K 2023-08-23 10:53:44,908 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 36ms, sequenceid=83, compaction requested=true 2023-08-23 10:53:44,908 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:44,908 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:44,910 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,910 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:44,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@343d3f3d[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:44,913 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:44,913 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:44,914 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,914 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:44,917 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,920 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,920 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,920 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:44,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:44,923 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:44,924 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:44,924 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:44,924 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,924 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,927 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,927 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:44,927 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,928 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,929 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,930 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:44,930 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,932 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,932 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,932 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:44,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:44,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:44,933 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:44,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:44,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:44,933 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:44,934 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:44,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:44,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:44,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,034 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:45,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:45,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:45,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:45,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,046 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:45,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:45,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:45,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,046 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,048 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,048 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,048 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,048 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,051 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,051 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,051 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,053 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,053 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:45,057 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,057 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,057 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,057 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-23 10:53:45,059 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,060 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,060 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,070 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,074 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,086 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,090 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,091 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/eaefbb847bd0460ebac659804557e1e3 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,095 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,095 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3, entries=5, sequenceid=91, filesize=5.0 K 2023-08-23 10:53:45,096 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/86fff6264dda4032878aa7dec7993b67 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,100 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,100 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67, entries=5, sequenceid=91, filesize=5.0 K 2023-08-23 10:53:45,101 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 44ms, sequenceid=91, compaction requested=true 2023-08-23 10:53:45,101 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:45,101 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,101 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,103 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38494517[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:45,105 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:45,106 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,107 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,108 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,109 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,110 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,110 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,110 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,111 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,111 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,111 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:45,114 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:45,114 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:45,114 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:45,114 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,114 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,116 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,117 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,117 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,117 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,119 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,127 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,128 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,128 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:45,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:45,127 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,128 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,128 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:45,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,229 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:45,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:45,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:53:45,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:53:45,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,239 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:45,239 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-08-23 10:53:45,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-08-23 10:53:45,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,241 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,242 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,242 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:53:45,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:53:45,244 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,244 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,244 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. started... 2023-08-23 10:53:45,245 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-08-23 10:53:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,248 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,248 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,248 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,248 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,248 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,254 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,258 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,267 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,270 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,271 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/d660326bf9914c87b1dbfe5c6a08955e as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,275 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,275 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e, entries=5, sequenceid=99, filesize=5.0 K 2023-08-23 10:53:45,275 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/65c6c4831514468f9555d18ec249e5eb as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,279 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,279 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb, entries=5, sequenceid=99, filesize=5.0 K 2023-08-23 10:53:45,280 INFO [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 35ms, sequenceid=99, compaction requested=true 2023-08-23 10:53:45,280 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:45,280 DEBUG [rs(jenkins-hbase4.apache.org,41821,1692787997489)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:53:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:53:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-08-23 10:53:45,280 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-08-23 10:53:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,282 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-08-23 10:53:45,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@698ddeab[Count = 0] remaining members to acquire global barrier 2023-08-23 10:53:45,284 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-08-23 10:53:45,284 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:53:45,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-08-23 10:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-08-23 10:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:53:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-08-23 10:53:45,294 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-08-23 10:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41821,1692787997489': 2023-08-23 10:53:45,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:53:45,297 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-08-23 10:53:45,297 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:53:45,297 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:53:45,297 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,297 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:53:45,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,300 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,302 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:53:45,302 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,302 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-08-23 10:53:45,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:53:45,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:53:45,305 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:53:45,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,306 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:45,406 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:53:45,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:53:45,409 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,410 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:45,411 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:45,411 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:45,411 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:45,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, UNASSIGN}] 2023-08-23 10:53:45,414 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, UNASSIGN 2023-08-23 10:53:45,414 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8cd300f31326ee729e983413d0ee732f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:45,414 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788025414"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788025414"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788025414"}]},"ts":"1692788025414"} 2023-08-23 10:53:45,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 8cd300f31326ee729e983413d0ee732f, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:45,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8cd300f31326ee729e983413d0ee732f, disabling compactions & flushes 2023-08-23 10:53:45,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. after waiting 0 ms 2023-08-23 10:53:45,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8cd300f31326ee729e983413d0ee732f 2/2 column families, dataSize=50 B heapSize=720 B 2023-08-23 10:53:45,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,593 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,594 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf1/388ff0c7ef8f45f09211b32966e93ace as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,598 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace, entries=1, sequenceid=103, filesize=4.9 K 2023-08-23 10:53:45,598 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.tmp/cf2/58640af2b78d44e880c9a15a44d38a0a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a, entries=1, sequenceid=103, filesize=4.9 K 2023-08-23 10:53:45,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 8cd300f31326ee729e983413d0ee732f in 35ms, sequenceid=103, compaction requested=true 2023-08-23 10:53:45,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-08-23 10:53:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:45,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f. 2023-08-23 10:53:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8cd300f31326ee729e983413d0ee732f: 2023-08-23 10:53:45,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=8cd300f31326ee729e983413d0ee732f, regionState=CLOSED 2023-08-23 10:53:45,610 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788025609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788025609"}]},"ts":"1692788025609"} 2023-08-23 10:53:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-08-23 10:53:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 8cd300f31326ee729e983413d0ee732f, server=jenkins-hbase4.apache.org,41821,1692787997489 in 196 msec 2023-08-23 10:53:45,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-08-23 10:53:45,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8cd300f31326ee729e983413d0ee732f, UNASSIGN in 199 msec 2023-08-23 10:53:45,634 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=8cd300f31326ee729e983413d0ee732f, threads=16 2023-08-23 10:53:45,634 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,634 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,634 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,634 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,635 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,637 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,637 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,637 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,640 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,641 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,641 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,641 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,641 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,643 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,644 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,645 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,652 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,653 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,654 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d, top=false 2023-08-23 10:53:45,657 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee, top=false 2023-08-23 10:53:45,657 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,657 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:45,658 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e, top=false 2023-08-23 10:53:45,659 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:45,661 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374, top=false 2023-08-23 10:53:45,663 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,672 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:45,672 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1, top=false 2023-08-23 10:53:45,672 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb, top=true 2023-08-23 10:53:45,673 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7, top=false 2023-08-23 10:53:45,680 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:45,680 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a, top=true 2023-08-23 10:53:45,681 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:45,681 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace, top=true 2023-08-23 10:53:45,682 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:45,682 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67, top=true 2023-08-23 10:53:45,717 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,721 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1 for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,721 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,721 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,724 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,724 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,727 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,731 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7 for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,731 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,732 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,733 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,733 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,733 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67 for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,733 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,735 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:45,735 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e, top=true 2023-08-23 10:53:45,747 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,748 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,749 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:45,751 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:45,751 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3, top=true 2023-08-23 10:53:45,759 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,759 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374 for child: e2d75dd262274c53186d4a8a3dc32e21, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,767 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,769 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,769 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,785 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3 for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,785 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,786 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/.splits/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e for child: 0c8db88bd15dc03c296a5c5fe5edfc04, parent: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,786 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,793 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:45,811 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,828 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,838 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,842 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,851 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,851 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,853 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,859 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24 for region: 8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:45,859 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 8cd300f31326ee729e983413d0ee732f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:53:45,898 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-23 10:53:45,900 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-08-23 10:53:45,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1692788025902"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1692788025902"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1692788025902"}]},"ts":"1692788025902"} 2023-08-23 10:53:45,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788025902"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788025902"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788025902"}]},"ts":"1692788025902"} 2023-08-23 10:53:45,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788025902"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788025902"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788025902"}]},"ts":"1692788025902"} 2023-08-23 10:53:45,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, ASSIGN}] 2023-08-23 10:53:45,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, ASSIGN 2023-08-23 10:53:45,907 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:45,908 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, ASSIGN 2023-08-23 10:53:45,909 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41821,1692787997489; forceNewPlan=false, retain=false 2023-08-23 10:53:46,057 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:53:46,059 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=e2d75dd262274c53186d4a8a3dc32e21, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026059"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788026059"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788026059"}]},"ts":"1692788026059"} 2023-08-23 10:53:46,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure e2d75dd262274c53186d4a8a3dc32e21, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:46,063 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=0c8db88bd15dc03c296a5c5fe5edfc04, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788026063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788026063"}]},"ts":"1692788026063"} 2023-08-23 10:53:46,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 0c8db88bd15dc03c296a5c5fe5edfc04, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:46,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e2d75dd262274c53186d4a8a3dc32e21, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.', STARTKEY => '', ENDKEY => '60'} 2023-08-23 10:53:46,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:46,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:46,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,218 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,219 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1 2023-08-23 10:53:46,219 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1 2023-08-23 10:53:46,220 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2d75dd262274c53186d4a8a3dc32e21 columnFamilyName cf1 2023-08-23 10:53:46,230 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-bottom 2023-08-23 10:53:46,233 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,235 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-bottom 2023-08-23 10:53:46,239 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-bottom 2023-08-23 10:53:46,243 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-bottom 2023-08-23 10:53:46,247 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374 2023-08-23 10:53:46,250 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7 2023-08-23 10:53:46,254 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1 2023-08-23 10:53:46,254 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(310): Store=e2d75dd262274c53186d4a8a3dc32e21/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:46,255 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,256 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2 2023-08-23 10:53:46,256 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2 2023-08-23 10:53:46,256 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e2d75dd262274c53186d4a8a3dc32e21 columnFamilyName cf2 2023-08-23 10:53:46,267 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-bottom 2023-08-23 10:53:46,272 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-bottom 2023-08-23 10:53:46,275 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,276 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-bottom 2023-08-23 10:53:46,281 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-bottom 2023-08-23 10:53:46,284 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d 2023-08-23 10:53:46,287 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e 2023-08-23 10:53:46,291 DEBUG [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee 2023-08-23 10:53:46,291 INFO [StoreOpener-e2d75dd262274c53186d4a8a3dc32e21-1] regionserver.HStore(310): Store=e2d75dd262274c53186d4a8a3dc32e21/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:46,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:46,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e2d75dd262274c53186d4a8a3dc32e21; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73939597, jitterRate=0.10178585350513458}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:46,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21., pid=77, masterSystemTime=1692788026213 2023-08-23 10:53:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,299 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:46,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:46,299 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:46,299 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,299 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,299 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1912): e2d75dd262274c53186d4a8a3dc32e21/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:46,299 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1912): e2d75dd262274c53186d4a8a3dc32e21/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:46,299 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e2d75dd262274c53186d4a8a3dc32e21/cf1 in testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,299 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e2d75dd262274c53186d4a8a3dc32e21/cf2 in testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,300 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp, totalSize=35.0 K 2023-08-23 10:53:46,300 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp, totalSize=35.0 K 2023-08-23 10:53:46,300 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788023729 2023-08-23 10:53:46,301 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting 0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788023729 2023-08-23 10:53:46,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,301 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788023920 2023-08-23 10:53:46,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0c8db88bd15dc03c296a5c5fe5edfc04, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.', STARTKEY => '60', ENDKEY => ''} 2023-08-23 10:53:46,301 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788023920 2023-08-23 10:53:46,301 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=e2d75dd262274c53186d4a8a3dc32e21, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,301 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788024106 2023-08-23 10:53:46,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026301"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788026301"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788026301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788026301"}]},"ts":"1692788026301"} 2023-08-23 10:53:46,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,302 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788024106 2023-08-23 10:53:46,302 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788024287 2023-08-23 10:53:46,303 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788024287 2023-08-23 10:53:46,303 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788024475 2023-08-23 10:53:46,303 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788024475 2023-08-23 10:53:46,304 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,304 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788024672 2023-08-23 10:53:46,304 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788024672 2023-08-23 10:53:46,305 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting 9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788024855 2023-08-23 10:53:46,305 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788024855 2023-08-23 10:53:46,305 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1 2023-08-23 10:53:46,305 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1 2023-08-23 10:53:46,306 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c8db88bd15dc03c296a5c5fe5edfc04 columnFamilyName cf1 2023-08-23 10:53:46,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-08-23 10:53:46,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure e2d75dd262274c53186d4a8a3dc32e21, server=jenkins-hbase4.apache.org,41821,1692787997489 in 242 msec 2023-08-23 10:53:46,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, ASSIGN in 401 msec 2023-08-23 10:53:46,319 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-top 2023-08-23 10:53:46,323 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,323 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-top 2023-08-23 10:53:46,324 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): e2d75dd262274c53186d4a8a3dc32e21#cf2#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,326 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): e2d75dd262274c53186d4a8a3dc32e21#cf1#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,326 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-top 2023-08-23 10:53:46,330 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-top 2023-08-23 10:53:46,334 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:46,334 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:46,337 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c 2023-08-23 10:53:46,338 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:46,338 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:46,341 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp/cf1/c77d0adf0ec6428ca8351fb76fa3987d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/c77d0adf0ec6428ca8351fb76fa3987d 2023-08-23 10:53:46,344 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:46,344 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:46,344 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(310): Store=0c8db88bd15dc03c296a5c5fe5edfc04/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:46,344 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,345 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e2d75dd262274c53186d4a8a3dc32e21/cf2 of e2d75dd262274c53186d4a8a3dc32e21 into 3b06ad3fe4d4435ba2cfab7e2bd2776c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,345 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,345 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21., storeName=e2d75dd262274c53186d4a8a3dc32e21/cf2, priority=9, startTime=1692788026299; duration=0sec 2023-08-23 10:53:46,345 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,345 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2 2023-08-23 10:53:46,345 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2 2023-08-23 10:53:46,345 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c8db88bd15dc03c296a5c5fe5edfc04 columnFamilyName cf2 2023-08-23 10:53:46,346 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e2d75dd262274c53186d4a8a3dc32e21/cf1 of e2d75dd262274c53186d4a8a3dc32e21 into c77d0adf0ec6428ca8351fb76fa3987d(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,346 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,346 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21., storeName=e2d75dd262274c53186d4a8a3dc32e21/cf1, priority=9, startTime=1692788026298; duration=0sec 2023-08-23 10:53:46,346 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,358 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-top 2023-08-23 10:53:46,361 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-top 2023-08-23 10:53:46,365 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,365 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-top 2023-08-23 10:53:46,369 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-top 2023-08-23 10:53:46,372 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:46,372 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:46,376 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:46,376 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:46,379 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:46,379 DEBUG [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:46,379 INFO [StoreOpener-0c8db88bd15dc03c296a5c5fe5edfc04-1] regionserver.HStore(310): Store=0c8db88bd15dc03c296a5c5fe5edfc04/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:46,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:53:46,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0c8db88bd15dc03c296a5c5fe5edfc04; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60352273, jitterRate=-0.10068105161190033}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:53:46,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04., pid=78, masterSystemTime=1692788026213 2023-08-23 10:53:46,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,385 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:46,385 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:53:46,385 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:46,386 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,386 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1912): 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 is initiating minor compaction (all files) 2023-08-23 10:53:46,386 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1912): 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 is initiating minor compaction (all files) 2023-08-23 10:53:46,386 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,386 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,386 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp, totalSize=34.8 K 2023-08-23 10:53:46,386 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp, totalSize=34.8 K 2023-08-23 10:53:46,386 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting 0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788023729 2023-08-23 10:53:46,387 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788023729 2023-08-23 10:53:46,387 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788023920 2023-08-23 10:53:46,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,387 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788023920 2023-08-23 10:53:46,388 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788024106 2023-08-23 10:53:46,388 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=0c8db88bd15dc03c296a5c5fe5edfc04, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,388 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788024106 2023-08-23 10:53:46,388 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1692788024855 2023-08-23 10:53:46,388 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026388"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788026388"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788026388"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788026388"}]},"ts":"1692788026388"} 2023-08-23 10:53:46,388 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting 9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1692788024855 2023-08-23 10:53:46,389 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,389 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,389 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,389 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,390 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,390 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-08-23 10:53:46,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 0c8db88bd15dc03c296a5c5fe5edfc04, server=jenkins-hbase4.apache.org,41821,1692787997489 in 324 msec 2023-08-23 10:53:46,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-08-23 10:53:46,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, ASSIGN in 486 msec 2023-08-23 10:53:46,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 in 983 msec 2023-08-23 10:53:46,402 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:53:46,407 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c8db88bd15dc03c296a5c5fe5edfc04#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,412 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c8db88bd15dc03c296a5c5fe5edfc04#cf1#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,443 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3cbc9e86cfb14c7e9ebcfb1062489b9d: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,443 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 317f380acec44ede9a5d77342de0b852: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,443 DEBUG [Listener at localhost/34585] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-08-23 10:53:46,444 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d 2023-08-23 10:53:46,444 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp/cf1/317f380acec44ede9a5d77342de0b852 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/317f380acec44ede9a5d77342de0b852 2023-08-23 10:53:46,449 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 317f380acec44ede9a5d77342de0b852: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,449 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3cbc9e86cfb14c7e9ebcfb1062489b9d: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,451 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 of 0c8db88bd15dc03c296a5c5fe5edfc04 into 317f380acec44ede9a5d77342de0b852(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,451 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,451 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04., storeName=0c8db88bd15dc03c296a5c5fe5edfc04/cf1, priority=9, startTime=1692788026384; duration=0sec 2023-08-23 10:53:46,451 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,451 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 of 0c8db88bd15dc03c296a5c5fe5edfc04 into 3cbc9e86cfb14c7e9ebcfb1062489b9d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,451 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,451 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04., storeName=0c8db88bd15dc03c296a5c5fe5edfc04/cf2, priority=9, startTime=1692788026385; duration=0sec 2023-08-23 10:53:46,451 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,458 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39252, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-08-23 10:53:46,458 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,459 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-23 10:53:46,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.HStore(1912): e2d75dd262274c53186d4a8a3dc32e21/cf1 is initiating major compaction (all files) 2023-08-23 10:53:46,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f03b60d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,460 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e2d75dd262274c53186d4a8a3dc32e21/cf1 in testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-23 10:53:46,460 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/c77d0adf0ec6428ca8351fb76fa3987d] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp, totalSize=6.8 K 2023-08-23 10:53:46,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.HStore(1912): e2d75dd262274c53186d4a8a3dc32e21/cf2 is initiating major compaction (all files) 2023-08-23 10:53:46,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@59024136; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:46,460 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e2d75dd262274c53186d4a8a3dc32e21/cf2 in testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,460 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp, totalSize=6.8 K 2023-08-23 10:53:46,461 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting c77d0adf0ec6428ca8351fb76fa3987d, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788023729 2023-08-23 10:53:46,461 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 3b06ad3fe4d4435ba2cfab7e2bd2776c, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788023729 2023-08-23 10:53:46,463 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.HStore(1912): 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 is initiating major compaction (all files) 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@cf44ecf; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.HStore(1912): 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 is initiating major compaction (all files) 2023-08-23 10:53:46,463 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@70b6d29a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-23 10:53:46,464 INFO [Listener at localhost/34585] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-08-23 10:53:46,469 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): e2d75dd262274c53186d4a8a3dc32e21#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,469 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): e2d75dd262274c53186d4a8a3dc32e21#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. is not mergeable because it has references 2023-08-23 10:53:46,481 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. is not mergeable because it has references 2023-08-23 10:53:46,501 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp/cf2/cfbdbfaf7771432abd109f7c2b321eed as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/cfbdbfaf7771432abd109f7c2b321eed 2023-08-23 10:53:46,501 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/.tmp/cf1/b82bce6495234384a07016fe72d6df92 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/b82bce6495234384a07016fe72d6df92 2023-08-23 10:53:46,507 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e2d75dd262274c53186d4a8a3dc32e21/cf1 of e2d75dd262274c53186d4a8a3dc32e21 into b82bce6495234384a07016fe72d6df92(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,507 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e2d75dd262274c53186d4a8a3dc32e21/cf2 of e2d75dd262274c53186d4a8a3dc32e21 into cfbdbfaf7771432abd109f7c2b321eed(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,507 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,507 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1692788026460; duration=0sec 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1692788026460; duration=0sec 2023-08-23 10:53:46,509 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-23 10:53:46,509 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/317f380acec44ede9a5d77342de0b852] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp, totalSize=5.0 K 2023-08-23 10:53:46,509 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp, totalSize=5.0 K 2023-08-23 10:53:46,510 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] compactions.Compactor(207): Compacting 317f380acec44ede9a5d77342de0b852, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,510 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] compactions.Compactor(207): Compacting 3cbc9e86cfb14c7e9ebcfb1062489b9d, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-08-23 10:53:46,513 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c8db88bd15dc03c296a5c5fe5edfc04#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,513 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c8db88bd15dc03c296a5c5fe5edfc04#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:53:46,532 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4c900c688d1d43688b2bb8f6469a3baa: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,533 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp/cf2/4c900c688d1d43688b2bb8f6469a3baa as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/4c900c688d1d43688b2bb8f6469a3baa 2023-08-23 10:53:46,537 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4c900c688d1d43688b2bb8f6469a3baa: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,538 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 0c8db88bd15dc03c296a5c5fe5edfc04/cf2 of 0c8db88bd15dc03c296a5c5fe5edfc04 into 4c900c688d1d43688b2bb8f6469a3baa(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,539 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,539 INFO [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1692788026463; duration=0sec 2023-08-23 10:53:46,539 DEBUG [RS:1;jenkins-hbase4:41821-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,547 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 821977a592364f78998ad6df51594993: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,548 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/.tmp/cf1/821977a592364f78998ad6df51594993 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/821977a592364f78998ad6df51594993 2023-08-23 10:53:46,552 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 821977a592364f78998ad6df51594993: NONE, but ROW specified in column family configuration 2023-08-23 10:53:46,553 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 0c8db88bd15dc03c296a5c5fe5edfc04/cf1 of 0c8db88bd15dc03c296a5c5fe5edfc04 into 821977a592364f78998ad6df51594993(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:53:46,553 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,553 INFO [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1692788026463; duration=0sec 2023-08-23 10:53:46,553 DEBUG [RS:1;jenkins-hbase4:41821-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:53:46,589 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41821] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. is not mergeable because it has references 2023-08-23 10:53:46,590 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41821] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. is not mergeable because it has references 2023-08-23 10:53:46,596 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-08-23 10:53:46,597 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:46,599 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-23 10:53:46,603 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788026603"}]},"ts":"1692788026603"} 2023-08-23 10:53:46,604 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-08-23 10:53:46,606 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-08-23 10:53:46,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, UNASSIGN}] 2023-08-23 10:53:46,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, UNASSIGN 2023-08-23 10:53:46,608 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=0c8db88bd15dc03c296a5c5fe5edfc04, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,608 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788026608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788026608"}]},"ts":"1692788026608"} 2023-08-23 10:53:46,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 0c8db88bd15dc03c296a5c5fe5edfc04, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:46,611 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, UNASSIGN 2023-08-23 10:53:46,612 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e2d75dd262274c53186d4a8a3dc32e21, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:53:46,612 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026612"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788026612"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788026612"}]},"ts":"1692788026612"} 2023-08-23 10:53:46,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure e2d75dd262274c53186d4a8a3dc32e21, server=jenkins-hbase4.apache.org,41821,1692787997489}] 2023-08-23 10:53:46,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-23 10:53:46,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0c8db88bd15dc03c296a5c5fe5edfc04, disabling compactions & flushes 2023-08-23 10:53:46,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. after waiting 0 ms 2023-08-23 10:53:46,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/317f380acec44ede9a5d77342de0b852, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace] to archive 2023-08-23 10:53:46,762 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:46,764 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,765 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,766 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,768 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,769 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:46,770 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:46,771 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/317f380acec44ede9a5d77342de0b852 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/317f380acec44ede9a5d77342de0b852 2023-08-23 10:53:46,772 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:46,774 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a] to archive 2023-08-23 10:53:46,774 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:46,776 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,777 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,778 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,779 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,780 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:46,781 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:46,782 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/3cbc9e86cfb14c7e9ebcfb1062489b9d 2023-08-23 10:53:46,783 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:46,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-08-23 10:53:46,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:46,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04. 2023-08-23 10:53:46,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0c8db88bd15dc03c296a5c5fe5edfc04: 2023-08-23 10:53:46,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e2d75dd262274c53186d4a8a3dc32e21, disabling compactions & flushes 2023-08-23 10:53:46,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. after waiting 0 ms 2023-08-23 10:53:46,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,789 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=0c8db88bd15dc03c296a5c5fe5edfc04, regionState=CLOSED 2023-08-23 10:53:46,789 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788026789"}]},"ts":"1692788026789"} 2023-08-23 10:53:46,790 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/c77d0adf0ec6428ca8351fb76fa3987d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c-bottom] to archive 2023-08-23 10:53:46,791 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:46,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-08-23 10:53:46,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 0c8db88bd15dc03c296a5c5fe5edfc04, server=jenkins-hbase4.apache.org,41821,1692787997489 in 182 msec 2023-08-23 10:53:46,793 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/0f1f408d6ff540569144304835bc4fd7.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=0c8db88bd15dc03c296a5c5fe5edfc04, UNASSIGN in 186 msec 2023-08-23 10:53:46,794 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/a4071ad9fc9b49c998227632a7831a72.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,795 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/fe55f2095c024cdfadb9d8c784fb5f24.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,796 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-9dfbbe9fac5c4a57a67a01216e296ac1 2023-08-23 10:53:46,797 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-4dfe91412d1f4e6fa9832019f87732c7 2023-08-23 10:53:46,797 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-0a934819e8dd43d7b50d648d80e3f374 2023-08-23 10:53:46,798 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/c77d0adf0ec6428ca8351fb76fa3987d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/c77d0adf0ec6428ca8351fb76fa3987d 2023-08-23 10:53:46,799 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/9383c22c8ee8467a8b5af802cb00a47c.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,801 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597-bottom] to archive 2023-08-23 10:53:46,801 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:53:46,803 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1b424ec7a42047f4889c49c3b7a7ed98.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,804 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/ccc5695eba4d46f3b9578112b9b6c175.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,805 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/1ef3b476515e44c4b4ff8c81cc0f5700.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,806 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-a8036d1b54ea4101a1e8cd70d83768ee 2023-08-23 10:53:46,806 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-857e7eb8324d4574a7855440d98b7c4e 2023-08-23 10:53:46,807 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/testSplitTableRegionDeletedRowsDaughter=8cd300f31326ee729e983413d0ee732f-5db80d0ecff54b40b72a6e42d338be1d 2023-08-23 10:53:46,808 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/3b06ad3fe4d4435ba2cfab7e2bd2776c 2023-08-23 10:53:46,809 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/78c0e77ee34f48efb9d20b8b2f9ea597.8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-08-23 10:53:46,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:46,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21. 2023-08-23 10:53:46,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e2d75dd262274c53186d4a8a3dc32e21: 2023-08-23 10:53:46,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e2d75dd262274c53186d4a8a3dc32e21, regionState=CLOSED 2023-08-23 10:53:46,815 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1692788026815"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788026815"}]},"ts":"1692788026815"} 2023-08-23 10:53:46,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-08-23 10:53:46,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure e2d75dd262274c53186d4a8a3dc32e21, server=jenkins-hbase4.apache.org,41821,1692787997489 in 203 msec 2023-08-23 10:53:46,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-08-23 10:53:46,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e2d75dd262274c53186d4a8a3dc32e21, UNASSIGN in 212 msec 2023-08-23 10:53:46,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788026820"}]},"ts":"1692788026820"} 2023-08-23 10:53:46,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-08-23 10:53:46,822 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-08-23 10:53:46,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 224 msec 2023-08-23 10:53:46,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-08-23 10:53:46,905 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-08-23 10:53:46,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,908 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,909 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-23 10:53:46,913 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,913 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,913 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,915 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/recovered.edits] 2023-08-23 10:53:46,915 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/recovered.edits] 2023-08-23 10:53:46,915 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/recovered.edits] 2023-08-23 10:53:46,919 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/821977a592364f78998ad6df51594993 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf1/821977a592364f78998ad6df51594993 2023-08-23 10:53:46,919 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/b82bce6495234384a07016fe72d6df92 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf1/b82bce6495234384a07016fe72d6df92 2023-08-23 10:53:46,919 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0a934819e8dd43d7b50d648d80e3f374 2023-08-23 10:53:46,921 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/0f1f408d6ff540569144304835bc4fd7 2023-08-23 10:53:46,922 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/4c900c688d1d43688b2bb8f6469a3baa to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/cf2/4c900c688d1d43688b2bb8f6469a3baa 2023-08-23 10:53:46,922 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/cfbdbfaf7771432abd109f7c2b321eed to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/cf2/cfbdbfaf7771432abd109f7c2b321eed 2023-08-23 10:53:46,923 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/388ff0c7ef8f45f09211b32966e93ace 2023-08-23 10:53:46,925 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/4dfe91412d1f4e6fa9832019f87732c7 2023-08-23 10:53:46,926 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/recovered.edits/113.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04/recovered.edits/113.seqid 2023-08-23 10:53:46,926 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/recovered.edits/113.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21/recovered.edits/113.seqid 2023-08-23 10:53:46,926 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9383c22c8ee8467a8b5af802cb00a47c 2023-08-23 10:53:46,927 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:53:46,927 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e2d75dd262274c53186d4a8a3dc32e21 2023-08-23 10:53:46,928 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/9dfbbe9fac5c4a57a67a01216e296ac1 2023-08-23 10:53:46,929 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/a4071ad9fc9b49c998227632a7831a72 2023-08-23 10:53:46,930 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/d660326bf9914c87b1dbfe5c6a08955e 2023-08-23 10:53:46,931 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/eaefbb847bd0460ebac659804557e1e3 2023-08-23 10:53:46,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf1/fe55f2095c024cdfadb9d8c784fb5f24 2023-08-23 10:53:46,936 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1b424ec7a42047f4889c49c3b7a7ed98 2023-08-23 10:53:46,937 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/1ef3b476515e44c4b4ff8c81cc0f5700 2023-08-23 10:53:46,938 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/58640af2b78d44e880c9a15a44d38a0a 2023-08-23 10:53:46,939 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/5db80d0ecff54b40b72a6e42d338be1d 2023-08-23 10:53:46,941 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/65c6c4831514468f9555d18ec249e5eb 2023-08-23 10:53:46,942 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/78c0e77ee34f48efb9d20b8b2f9ea597 2023-08-23 10:53:46,943 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/857e7eb8324d4574a7855440d98b7c4e 2023-08-23 10:53:46,944 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/86fff6264dda4032878aa7dec7993b67 2023-08-23 10:53:46,945 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/a8036d1b54ea4101a1e8cd70d83768ee 2023-08-23 10:53:46,946 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/cf2/ccc5695eba4d46f3b9578112b9b6c175 2023-08-23 10:53:46,949 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/recovered.edits/106.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f/recovered.edits/106.seqid 2023-08-23 10:53:46,949 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8cd300f31326ee729e983413d0ee732f 2023-08-23 10:53:46,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-08-23 10:53:46,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,953 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-08-23 10:53:46,956 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-08-23 10:53:46,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-08-23 10:53:46,957 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788026957"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:46,957 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788026957"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:46,957 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788026957"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:46,959 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:53:46,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8cd300f31326ee729e983413d0ee732f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1692788023225.8cd300f31326ee729e983413d0ee732f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e2d75dd262274c53186d4a8a3dc32e21, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1692788025409.e2d75dd262274c53186d4a8a3dc32e21.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 0c8db88bd15dc03c296a5c5fe5edfc04, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1692788025409.0c8db88bd15dc03c296a5c5fe5edfc04.', STARTKEY => '60', ENDKEY => ''}] 2023-08-23 10:53:46,959 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-08-23 10:53:46,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788026959"}]},"ts":"9223372036854775807"} 2023-08-23 10:53:46,961 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-08-23 10:53:46,963 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:53:46,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 57 msec 2023-08-23 10:53:47,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-08-23 10:53:47,010 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-08-23 10:53:47,025 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 409) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:58894 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:44470 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:38038 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39270 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:44458 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:54428 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:39298 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:44450 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:45978 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:54580 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:37962 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:37948 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c60eea6-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:54630 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:53704 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:50306 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-240906504_17 at /127.0.0.1:53416 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=872 (was 868) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 300), ProcessCount=171 (was 171), AvailableMemoryMB=10844 (was 10854) 2023-08-23 10:53:47,039 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=872, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=171, AvailableMemoryMB=10843 2023-08-23 10:53:47,039 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:53:47,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:53:47,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:53:47,046 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:53:47,048 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:53:47,049 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:53:47,050 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,051 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf empty. 2023-08-23 10:53:47,051 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,051 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-23 10:53:47,066 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:47,067 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 52c62d6f8ed74063b747c3f1cbbc4ccf, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 52c62d6f8ed74063b747c3f1cbbc4ccf, disabling compactions & flushes 2023-08-23 10:53:47,077 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. after waiting 0 ms 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,077 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,077 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 52c62d6f8ed74063b747c3f1cbbc4ccf: 2023-08-23 10:53:47,079 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:53:47,080 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788027080"}]},"ts":"1692788027080"} 2023-08-23 10:53:47,081 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:53:47,082 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:53:47,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788027082"}]},"ts":"1692788027082"} 2023-08-23 10:53:47,083 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-08-23 10:53:47,086 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:47,087 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:47,087 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:47,087 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:47,087 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:53:47,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, ASSIGN}] 2023-08-23 10:53:47,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, ASSIGN 2023-08-23 10:53:47,089 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:53:47,239 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:47,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:47,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027242"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788027242"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788027242"}]},"ts":"1692788027242"} 2023-08-23 10:53:47,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:47,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 52c62d6f8ed74063b747c3f1cbbc4ccf, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:47,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:47,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:47,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,400 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,401 DEBUG [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:53:47,401 DEBUG [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:53:47,402 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52c62d6f8ed74063b747c3f1cbbc4ccf columnFamilyName cf1 2023-08-23 10:53:47,402 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] regionserver.HStore(310): Store=52c62d6f8ed74063b747c3f1cbbc4ccf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:47,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:53:47,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 52c62d6f8ed74063b747c3f1cbbc4ccf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63514908, jitterRate=-0.05355411767959595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-23 10:53:47,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 52c62d6f8ed74063b747c3f1cbbc4ccf: 2023-08-23 10:53:47,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf., pid=87, masterSystemTime=1692788027395 2023-08-23 10:53:47,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:47,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027409"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788027409"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788027409"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788027409"}]},"ts":"1692788027409"} 2023-08-23 10:53:47,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-08-23 10:53:47,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532 in 166 msec 2023-08-23 10:53:47,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-08-23 10:53:47,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, ASSIGN in 325 msec 2023-08-23 10:53:47,414 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:53:47,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788027414"}]},"ts":"1692788027414"} 2023-08-23 10:53:47,415 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-08-23 10:53:47,425 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:53:47,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 380 msec 2023-08-23 10:53:47,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-08-23 10:53:47,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:53:47,577 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-23 10:53:47,578 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-08-23 10:53:47,578 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-08-23 10:53:47,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-08-23 10:53:47,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, REOPEN/MOVE}] 2023-08-23 10:53:47,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, REOPEN/MOVE 2023-08-23 10:53:47,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:47,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788027595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788027595"}]},"ts":"1692788027595"} 2023-08-23 10:53:47,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:47,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 52c62d6f8ed74063b747c3f1cbbc4ccf, disabling compactions & flushes 2023-08-23 10:53:47,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. after waiting 0 ms 2023-08-23 10:53:47,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-08-23 10:53:47,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:53:47,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:47,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 52c62d6f8ed74063b747c3f1cbbc4ccf: 2023-08-23 10:53:47,754 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 52c62d6f8ed74063b747c3f1cbbc4ccf to self. 2023-08-23 10:53:47,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:47,755 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=CLOSED 2023-08-23 10:53:47,756 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788027755"}]},"ts":"1692788027755"} 2023-08-23 10:53:47,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-08-23 10:53:47,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532 in 161 msec 2023-08-23 10:53:47,759 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=true 2023-08-23 10:53:47,909 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:53:47,909 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:47,910 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788027909"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788027909"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788027909"}]},"ts":"1692788027909"} 2023-08-23 10:53:47,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:53:48,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:48,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 52c62d6f8ed74063b747c3f1cbbc4ccf, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:53:48,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:48,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:48,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,068 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,069 DEBUG [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:53:48,069 DEBUG [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:53:48,069 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52c62d6f8ed74063b747c3f1cbbc4ccf columnFamilyName cf1 2023-08-23 10:53:48,070 INFO [StoreOpener-52c62d6f8ed74063b747c3f1cbbc4ccf-1] regionserver.HStore(310): Store=52c62d6f8ed74063b747c3f1cbbc4ccf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:53:48,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:53:48,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 52c62d6f8ed74063b747c3f1cbbc4ccf; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72750921, jitterRate=0.08407320082187653}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-23 10:53:48,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 52c62d6f8ed74063b747c3f1cbbc4ccf: 2023-08-23 10:53:48,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf., pid=92, masterSystemTime=1692788028063 2023-08-23 10:53:48,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:48,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:53:48,077 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:53:48,077 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788028077"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788028077"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788028077"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788028077"}]},"ts":"1692788028077"} 2023-08-23 10:53:48,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-08-23 10:53:48,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532 in 167 msec 2023-08-23 10:53:48,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-08-23 10:53:48,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, REOPEN/MOVE in 487 msec 2023-08-23 10:53:48,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-08-23 10:53:48,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-08-23 10:53:48,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, ASSIGN}] 2023-08-23 10:53:48,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, ASSIGN 2023-08-23 10:53:48,088 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-08-23 10:53:48,239 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:53:48,240 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:53:48,240 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:53:48,240 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:53:48,240 DEBUG [jenkins-hbase4:44301] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-08-23 10:53:48,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=fa325bce8f770acd7b48d1ba76647d7b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:53:48,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788028241"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1692788028241"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1692788028241"}]},"ts":"1692788028241"} 2023-08-23 10:53:48,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure fa325bce8f770acd7b48d1ba76647d7b, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:53:48,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:53:48,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fa325bce8f770acd7b48d1ba76647d7b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-08-23 10:53:48,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:53:48,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:53:48,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:53:48,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:53:48,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:53:48,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:48,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:48,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:48,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:48,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:48,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,065 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-08-23 10:53:49,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:49,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:50,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,718 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:53:51,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:51,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:52,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:53,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:54,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:55,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:56,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:57,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:58,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:53:59,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:00,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:01,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:02,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:03,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:04,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:05,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:06,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:07,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:08,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:09,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:10,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:11,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:12,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:13,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:14,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:15,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:16,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:17,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:18,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:19,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:20,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:22,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:23,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:24,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:25,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:26,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:27,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:28,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:29,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:30,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:31,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:32,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:33,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:34,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:35,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:36,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:37,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:38,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:39,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:40,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:41,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:42,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:43,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:44,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:45,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:46,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-23 10:54:47,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-23 10:54:47,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:47,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-23 10:54:48,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-23 10:54:48,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-08-23 10:54:48,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:48,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,781 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:49,783 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:49,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:49,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:49,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:49,786 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 52c62d6f8ed74063b747c3f1cbbc4ccf, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:49,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 exec-time=5 msec 2023-08-23 10:54:49,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:49,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:50,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-08-23 10:54:50,131 INFO [StoreOpener-fa325bce8f770acd7b48d1ba76647d7b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:54:50,134 DEBUG [StoreOpener-fa325bce8f770acd7b48d1ba76647d7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:54:50,134 DEBUG [StoreOpener-fa325bce8f770acd7b48d1ba76647d7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1 2023-08-23 10:54:50,135 INFO [StoreOpener-fa325bce8f770acd7b48d1ba76647d7b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fa325bce8f770acd7b48d1ba76647d7b columnFamilyName cf1 2023-08-23 10:54:50,136 INFO [StoreOpener-fa325bce8f770acd7b48d1ba76647d7b-1] regionserver.HStore(310): Store=fa325bce8f770acd7b48d1ba76647d7b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:54:50,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fa325bce8f770acd7b48d1ba76647d7b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71105764, jitterRate=0.0595584511756897}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-08-23 10:54:50,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fa325bce8f770acd7b48d1ba76647d7b: 2023-08-23 10:54:50,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b., pid=94, masterSystemTime=1692788028397 2023-08-23 10:54:50,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:50,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:50,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=fa325bce8f770acd7b48d1ba76647d7b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:50,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788090139"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1692788090139"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1692788090139"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1692788090139"}]},"ts":"1692788090139"} 2023-08-23 10:54:50,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-08-23 10:54:50,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure fa325bce8f770acd7b48d1ba76647d7b, server=jenkins-hbase4.apache.org,37675,1692787997438 in 1 mins, 1.898 sec 2023-08-23 10:54:50,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-08-23 10:54:50,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, ASSIGN in 1 mins, 2.057 sec 2023-08-23 10:54:50,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.584 sec 2023-08-23 10:54:50,147 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1692788027555","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62591,"client":"172.31.14.131:47618","queuetimems":0,"class":"HMaster"} 2023-08-23 10:54:50,147 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-08-23 10:54:52,033 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up 2023-08-23 10:54:52,035 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:54:52,037 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-23 10:54:52,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788092041"}]},"ts":"1692788092041"} 2023-08-23 10:54:52,042 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-08-23 10:54:52,043 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-08-23 10:54:52,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, UNASSIGN}] 2023-08-23 10:54:52,045 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, UNASSIGN 2023-08-23 10:54:52,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=fa325bce8f770acd7b48d1ba76647d7b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:52,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788092045"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1692788092045"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1692788092045"}]},"ts":"1692788092045"} 2023-08-23 10:54:52,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure fa325bce8f770acd7b48d1ba76647d7b, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:54:52,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, UNASSIGN 2023-08-23 10:54:52,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:52,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788092049"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788092049"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788092049"}]},"ts":"1692788092049"} 2023-08-23 10:54:52,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:54:52,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-23 10:54:52,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:54:52,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fa325bce8f770acd7b48d1ba76647d7b, disabling compactions & flushes 2023-08-23 10:54:52,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:52,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:52,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. after waiting 0 ms 2023-08-23 10:54:52,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:52,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:54:52,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b. 2023-08-23 10:54:52,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fa325bce8f770acd7b48d1ba76647d7b: 2023-08-23 10:54:52,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fa325bce8f770acd7b48d1ba76647d7b 2023-08-23 10:54:52,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:54:52,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 52c62d6f8ed74063b747c3f1cbbc4ccf, disabling compactions & flushes 2023-08-23 10:54:52,204 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=fa325bce8f770acd7b48d1ba76647d7b, regionState=CLOSED 2023-08-23 10:54:52,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:54:52,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:54:52,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. after waiting 0 ms 2023-08-23 10:54:52,204 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788092203"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1692788092203"}]},"ts":"1692788092203"} 2023-08-23 10:54:52,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:54:52,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-08-23 10:54:52,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure fa325bce8f770acd7b48d1ba76647d7b, server=jenkins-hbase4.apache.org,37675,1692787997438 in 158 msec 2023-08-23 10:54:52,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-08-23 10:54:52,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:54:52,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b, UNASSIGN in 165 msec 2023-08-23 10:54:52,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf. 2023-08-23 10:54:52,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 52c62d6f8ed74063b747c3f1cbbc4ccf: 2023-08-23 10:54:52,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:54:52,213 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=52c62d6f8ed74063b747c3f1cbbc4ccf, regionState=CLOSED 2023-08-23 10:54:52,213 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1692788092213"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788092213"}]},"ts":"1692788092213"} 2023-08-23 10:54:52,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-08-23 10:54:52,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 52c62d6f8ed74063b747c3f1cbbc4ccf, server=jenkins-hbase4.apache.org,43499,1692787997532 in 163 msec 2023-08-23 10:54:52,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-08-23 10:54:52,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=52c62d6f8ed74063b747c3f1cbbc4ccf, UNASSIGN in 172 msec 2023-08-23 10:54:52,218 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788092218"}]},"ts":"1692788092218"} 2023-08-23 10:54:52,219 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-08-23 10:54:52,222 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-08-23 10:54:52,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 185 msec 2023-08-23 10:54:52,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-08-23 10:54:52,343 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-08-23 10:54:52,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-23 10:54:52,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,352 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:54:52,353 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits] 2023-08-23 10:54:52,358 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits/7.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf/recovered.edits/7.seqid 2023-08-23 10:54:52,358 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/52c62d6f8ed74063b747c3f1cbbc4ccf 2023-08-23 10:54:52,358 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-08-23 10:54:52,361 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,363 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-08-23 10:54:52,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-08-23 10:54:52,366 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,366 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-08-23 10:54:52,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788092366"}]},"ts":"9223372036854775807"} 2023-08-23 10:54:52,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788092366"}]},"ts":"9223372036854775807"} 2023-08-23 10:54:52,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-08-23 10:54:52,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 52c62d6f8ed74063b747c3f1cbbc4ccf, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045.52c62d6f8ed74063b747c3f1cbbc4ccf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fa325bce8f770acd7b48d1ba76647d7b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1692788027045_0001.fa325bce8f770acd7b48d1ba76647d7b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-08-23 10:54:52,368 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-08-23 10:54:52,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788092368"}]},"ts":"9223372036854775807"} 2023-08-23 10:54:52,369 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-08-23 10:54:52,371 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:52,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 28 msec 2023-08-23 10:54:52,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-08-23 10:54:52,448 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-08-23 10:54:52,464 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=349 (was 414), OpenFileDescriptor=658 (was 872), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 300), ProcessCount=171 (was 171), AvailableMemoryMB=10831 (was 10843) 2023-08-23 10:54:52,480 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=350, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=171, AvailableMemoryMB=10831 2023-08-23 10:54:52,480 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:54:52,482 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:54:52,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:54:52,484 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:52,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:54:52,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:54:52,488 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,488 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 empty. 2023-08-23 10:54:52,489 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-23 10:54:52,501 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-08-23 10:54:52,502 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => c46b502b19bda6c6c44af6dcb42f2a62, NAME => 'testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:54:52,513 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:54:52,514 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing c46b502b19bda6c6c44af6dcb42f2a62, disabling compactions & flushes 2023-08-23 10:54:52,514 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,514 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,514 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. after waiting 0 ms 2023-08-23 10:54:52,514 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,514 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,514 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:52,516 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:54:52,517 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1692788092517"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788092517"}]},"ts":"1692788092517"} 2023-08-23 10:54:52,518 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:54:52,519 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:54:52,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788092519"}]},"ts":"1692788092519"} 2023-08-23 10:54:52,520 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-08-23 10:54:52,524 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:54:52,525 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:54:52,525 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:54:52,525 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:54:52,525 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:54:52,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, ASSIGN}] 2023-08-23 10:54:52,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, ASSIGN 2023-08-23 10:54:52,527 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37675,1692787997438; forceNewPlan=false, retain=false 2023-08-23 10:54:52,677 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:54:52,678 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c46b502b19bda6c6c44af6dcb42f2a62, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:52,679 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1692788092678"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788092678"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788092678"}]},"ts":"1692788092678"} 2023-08-23 10:54:52,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure c46b502b19bda6c6c44af6dcb42f2a62, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:54:52,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c46b502b19bda6c6c44af6dcb42f2a62, NAME => 'testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:54:52,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:54:52,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:54:52,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,837 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,838 DEBUG [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1 2023-08-23 10:54:52,839 DEBUG [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1 2023-08-23 10:54:52,839 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c46b502b19bda6c6c44af6dcb42f2a62 columnFamilyName cf1 2023-08-23 10:54:52,839 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] regionserver.HStore(310): Store=c46b502b19bda6c6c44af6dcb42f2a62/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:54:52,839 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,840 DEBUG [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2 2023-08-23 10:54:52,840 DEBUG [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2 2023-08-23 10:54:52,841 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c46b502b19bda6c6c44af6dcb42f2a62 columnFamilyName cf2 2023-08-23 10:54:52,841 INFO [StoreOpener-c46b502b19bda6c6c44af6dcb42f2a62-1] regionserver.HStore(310): Store=c46b502b19bda6c6c44af6dcb42f2a62/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:54:52,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:54:52,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:52,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:54:52,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c46b502b19bda6c6c44af6dcb42f2a62; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61713392, jitterRate=-0.0803987979888916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:54:52,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:52,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62., pid=104, masterSystemTime=1692788092832 2023-08-23 10:54:52,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=c46b502b19bda6c6c44af6dcb42f2a62, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:52,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1692788092850"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788092850"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788092850"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788092850"}]},"ts":"1692788092850"} 2023-08-23 10:54:52,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-08-23 10:54:52,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure c46b502b19bda6c6c44af6dcb42f2a62, server=jenkins-hbase4.apache.org,37675,1692787997438 in 171 msec 2023-08-23 10:54:52,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-08-23 10:54:52,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, ASSIGN in 328 msec 2023-08-23 10:54:52,855 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:54:52,856 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788092856"}]},"ts":"1692788092856"} 2023-08-23 10:54:52,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-08-23 10:54:52,859 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:54:52,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 376 msec 2023-08-23 10:54:52,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:52,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:52,993 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:52,993 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:52,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:52,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:52,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:52,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,995 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:52,995 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:52,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:52,995 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:52,996 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:52,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:52,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:52,996 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:52,996 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:52,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:52,998 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:52,998 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:52,998 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:52,999 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,000 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,012 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/a7821b00112f40bebfb6b6f95b165250 2023-08-23 10:54:53,029 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1 2023-08-23 10:54:53,032 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/a7821b00112f40bebfb6b6f95b165250 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/a7821b00112f40bebfb6b6f95b165250 2023-08-23 10:54:53,037 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/a7821b00112f40bebfb6b6f95b165250, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:54:53,037 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1 2023-08-23 10:54:53,042 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:54:53,042 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 43ms, sequenceid=6, compaction requested=false 2023-08-23 10:54:53,043 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-08-23 10:54:53,043 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,043 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,044 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,044 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,044 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,046 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,046 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,046 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,046 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44c68c6d[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,048 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,048 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,049 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,050 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,050 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,051 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,052 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,052 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,055 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,055 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,055 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,055 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,055 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,056 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,057 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,057 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,058 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,058 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,060 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,060 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,061 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,064 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,064 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,065 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,066 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,066 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,066 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,066 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,165 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:53,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:53,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:53,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:53,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:53,179 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:53,179 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,179 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:53,179 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:53,180 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,180 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,181 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,181 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:53,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,183 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:53,184 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,184 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:53,184 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:53,185 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,191 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,191 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,191 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,191 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,191 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,195 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/5ea0f223725b469eafefd8f53a906066 2023-08-23 10:54:53,206 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/e96b3fa6fc1e48e2987f1166ba78f070 2023-08-23 10:54:53,210 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/5ea0f223725b469eafefd8f53a906066 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066 2023-08-23 10:54:53,214 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:54:53,215 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/e96b3fa6fc1e48e2987f1166ba78f070 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070 2023-08-23 10:54:53,220 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:54:53,221 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 37ms, sequenceid=19, compaction requested=false 2023-08-23 10:54:53,221 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,221 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,223 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,223 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19df871f[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,226 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,226 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,227 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,228 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,228 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,229 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,229 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,229 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,231 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,232 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,236 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,236 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,236 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,236 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,236 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,237 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,237 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,238 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,238 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,239 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,240 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,241 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,242 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,242 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,246 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,247 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,248 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,247 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,248 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,348 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:53,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:53,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:53,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:53,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:53,364 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:53,364 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,364 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:53,364 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:53,364 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,364 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,367 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,367 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:53,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:53,369 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,369 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:53,369 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,373 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,373 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,383 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f 2023-08-23 10:54:53,399 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/54d30408b6bd4f57b683d171f8383296 2023-08-23 10:54:53,403 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f 2023-08-23 10:54:53,407 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:54:53,408 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/54d30408b6bd4f57b683d171f8383296 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/54d30408b6bd4f57b683d171f8383296 2023-08-23 10:54:53,411 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/54d30408b6bd4f57b683d171f8383296, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:54:53,413 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 44ms, sequenceid=32, compaction requested=true 2023-08-23 10:54:53,413 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,413 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,415 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,416 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,416 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bcc18ce[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,417 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,418 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,419 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,419 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,419 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,421 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,421 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,421 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,422 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,422 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,422 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,422 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,423 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,423 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,423 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,426 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,426 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,426 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,426 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,426 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,428 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,428 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,428 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,428 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,428 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,429 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,430 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,430 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,432 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,432 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,432 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,438 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,438 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,439 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,539 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:53,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:53,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:53,554 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:53,554 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,554 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:53,554 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:53,555 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,555 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,556 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,556 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,556 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,556 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,558 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,559 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,558 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,559 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:53,559 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,568 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/f2291ea7e9ca47dbbc4733044d2fae18 2023-08-23 10:54:53,579 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/831fdbf3415748318920aad512e8ce9d 2023-08-23 10:54:53,582 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/f2291ea7e9ca47dbbc4733044d2fae18 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/f2291ea7e9ca47dbbc4733044d2fae18 2023-08-23 10:54:53,586 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/f2291ea7e9ca47dbbc4733044d2fae18, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:54:53,587 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/831fdbf3415748318920aad512e8ce9d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/831fdbf3415748318920aad512e8ce9d 2023-08-23 10:54:53,590 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/831fdbf3415748318920aad512e8ce9d, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:54:53,591 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 32ms, sequenceid=45, compaction requested=true 2023-08-23 10:54:53,591 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,591 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,594 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cb3eb9a[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,596 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,596 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,598 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,598 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,599 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,599 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,600 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,601 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,603 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,606 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,607 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,607 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,607 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,607 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,608 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,608 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,608 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,608 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,609 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,609 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,610 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,610 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,612 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,612 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,613 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,616 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,616 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,617 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,618 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,618 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,717 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:53,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:53,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:53,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:53,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:53,731 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:53,731 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,731 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:53,731 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:53,732 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,732 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,733 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,733 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,733 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:53,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,735 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,735 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,735 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,735 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,736 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:53,736 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:53,736 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,736 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:53,736 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:53,738 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,750 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/7c83be074ee64d5e858eacc50625dc64 2023-08-23 10:54:53,761 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/383e511320764c1cb56315595c2cfa3f 2023-08-23 10:54:53,765 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/7c83be074ee64d5e858eacc50625dc64 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/7c83be074ee64d5e858eacc50625dc64 2023-08-23 10:54:53,769 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/7c83be074ee64d5e858eacc50625dc64, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:54:53,769 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/383e511320764c1cb56315595c2cfa3f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/383e511320764c1cb56315595c2cfa3f 2023-08-23 10:54:53,773 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/383e511320764c1cb56315595c2cfa3f, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:54:53,774 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 38ms, sequenceid=58, compaction requested=true 2023-08-23 10:54:53,774 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,774 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,775 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,775 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,776 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,776 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,776 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,776 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ce6a785[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,778 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,778 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,780 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,784 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,784 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,784 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,788 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,788 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,788 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,788 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,789 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,789 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,789 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,792 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,792 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,792 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,792 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,792 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,794 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,795 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,795 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,795 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,797 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,797 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,797 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,801 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,801 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,802 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,902 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:53,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:53,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:53,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:53,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-08-23 10:54:53,916 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-08-23 10:54:53,916 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,916 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-08-23 10:54:53,916 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-08-23 10:54:53,917 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,917 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,919 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,919 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,919 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:53,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,921 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-08-23 10:54:53,922 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:53,922 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:53,922 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,922 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. started... 2023-08-23 10:54:53,922 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,923 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,930 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/113261b41bb6408c97431bdd73306508 2023-08-23 10:54:53,936 ERROR [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 0db35eca-a409-482d-977c-53ac02140840) service to localhost/127.0.0.1:34277 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:54:53,942 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/d93a555199be4b358177158493a41499 2023-08-23 10:54:53,946 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/113261b41bb6408c97431bdd73306508 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/113261b41bb6408c97431bdd73306508 2023-08-23 10:54:53,950 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/113261b41bb6408c97431bdd73306508, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:54:53,951 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/d93a555199be4b358177158493a41499 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/d93a555199be4b358177158493a41499 2023-08-23 10:54:53,954 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/d93a555199be4b358177158493a41499, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:54:53,955 INFO [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 33ms, sequenceid=71, compaction requested=true 2023-08-23 10:54:53,955 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:53,955 DEBUG [rs(jenkins-hbase4.apache.org,37675,1692787997438)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-08-23 10:54:53,955 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-08-23 10:54:53,957 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,957 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,957 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,957 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-08-23 10:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79e42c79[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:53,959 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-08-23 10:54:53,959 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:53,961 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,962 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-08-23 10:54:53,962 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,964 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,964 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-08-23 10:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37675,1692787997438': 2023-08-23 10:54:53,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-08-23 10:54:53,967 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-08-23 10:54:53,967 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:53,967 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:53,967 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-08-23 10:54:53,967 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:53,969 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,970 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:53,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,970 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,971 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,973 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:53,973 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-08-23 10:54:53,973 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,976 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,976 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,976 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,977 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:53,976 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-08-23 10:54:53,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:53,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,976 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:53,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:53,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:53,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:53,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:53,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:53,977 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:53,978 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:53,978 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:53,978 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-08-23 10:54:53,978 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-08-23 10:54:54,078 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:54,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:54,085 INFO [Listener at localhost/34585] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:54,087 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.StoreUtils(129): cannot split hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066 because midkey is the same as first or last row 2023-08-23 10:54:54,087 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.StoreUtils(129): cannot split hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070 because midkey is the same as first or last row 2023-08-23 10:54:54,087 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HRegion(2745): Flushing c46b502b19bda6c6c44af6dcb42f2a62 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:54:54,096 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/2947183121db40b08f9e88cc961f2bc0 2023-08-23 10:54:54,107 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/92a9ead8a95b4d56a69473f896fd4f7a 2023-08-23 10:54:54,111 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf1/2947183121db40b08f9e88cc961f2bc0 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/2947183121db40b08f9e88cc961f2bc0 2023-08-23 10:54:54,114 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/2947183121db40b08f9e88cc961f2bc0, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:54:54,115 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/.tmp/cf2/92a9ead8a95b4d56a69473f896fd4f7a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/92a9ead8a95b4d56a69473f896fd4f7a 2023-08-23 10:54:54,119 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/92a9ead8a95b4d56a69473f896fd4f7a, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:54:54,120 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c46b502b19bda6c6c44af6dcb42f2a62 in 33ms, sequenceid=82, compaction requested=true 2023-08-23 10:54:54,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.HRegion(2446): Flush status journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:54,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.StoreUtils(129): cannot split hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066 because midkey is the same as first or last row 2023-08-23 10:54:54,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37675] regionserver.StoreUtils(129): cannot split hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070 because midkey is the same as first or last row 2023-08-23 10:54:54,121 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:54,121 DEBUG [Listener at localhost/34585] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-08-23 10:54:54,122 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:54:54,123 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-08-23 10:54:54,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-08-23 10:54:54,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-23 10:54:54,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788094126"}]},"ts":"1692788094126"} 2023-08-23 10:54:54,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-08-23 10:54:54,129 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-08-23 10:54:54,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, UNASSIGN}] 2023-08-23 10:54:54,130 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, UNASSIGN 2023-08-23 10:54:54,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c46b502b19bda6c6c44af6dcb42f2a62, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:54,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1692788094130"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788094130"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788094130"}]},"ts":"1692788094130"} 2023-08-23 10:54:54,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure c46b502b19bda6c6c44af6dcb42f2a62, server=jenkins-hbase4.apache.org,37675,1692787997438}] 2023-08-23 10:54:54,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-23 10:54:54,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:54,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c46b502b19bda6c6c44af6dcb42f2a62, disabling compactions & flushes 2023-08-23 10:54:54,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:54,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:54,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. after waiting 0 ms 2023-08-23 10:54:54,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:54,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:54:54,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:54:54,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62. 2023-08-23 10:54:54,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c46b502b19bda6c6c44af6dcb42f2a62: 2023-08-23 10:54:54,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:54,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c46b502b19bda6c6c44af6dcb42f2a62, regionState=CLOSED 2023-08-23 10:54:54,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1692788094290"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788094290"}]},"ts":"1692788094290"} 2023-08-23 10:54:54,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-08-23 10:54:54,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure c46b502b19bda6c6c44af6dcb42f2a62, server=jenkins-hbase4.apache.org,37675,1692787997438 in 160 msec 2023-08-23 10:54:54,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-08-23 10:54:54,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=c46b502b19bda6c6c44af6dcb42f2a62, UNASSIGN in 164 msec 2023-08-23 10:54:54,294 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788094294"}]},"ts":"1692788094294"} 2023-08-23 10:54:54,295 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-08-23 10:54:54,297 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-08-23 10:54:54,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 174 msec 2023-08-23 10:54:54,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-08-23 10:54:54,428 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-08-23 10:54:54,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-08-23 10:54:54,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,431 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-23 10:54:54,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,435 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:54,437 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/recovered.edits] 2023-08-23 10:54:54,440 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/113261b41bb6408c97431bdd73306508 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/113261b41bb6408c97431bdd73306508 2023-08-23 10:54:54,442 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/16cd6cd98dd44cd0a6be19ee826f5e4f 2023-08-23 10:54:54,442 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/2947183121db40b08f9e88cc961f2bc0 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/2947183121db40b08f9e88cc961f2bc0 2023-08-23 10:54:54,443 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/5ea0f223725b469eafefd8f53a906066 2023-08-23 10:54:54,445 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/7c83be074ee64d5e858eacc50625dc64 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/7c83be074ee64d5e858eacc50625dc64 2023-08-23 10:54:54,446 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/a7821b00112f40bebfb6b6f95b165250 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/a7821b00112f40bebfb6b6f95b165250 2023-08-23 10:54:54,447 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/f2291ea7e9ca47dbbc4733044d2fae18 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf1/f2291ea7e9ca47dbbc4733044d2fae18 2023-08-23 10:54:54,449 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/30e1a2a4c9ba4d6c8b5a7294f993ffb1 2023-08-23 10:54:54,450 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/383e511320764c1cb56315595c2cfa3f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/383e511320764c1cb56315595c2cfa3f 2023-08-23 10:54:54,451 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/54d30408b6bd4f57b683d171f8383296 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/54d30408b6bd4f57b683d171f8383296 2023-08-23 10:54:54,452 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/831fdbf3415748318920aad512e8ce9d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/831fdbf3415748318920aad512e8ce9d 2023-08-23 10:54:54,453 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/92a9ead8a95b4d56a69473f896fd4f7a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/92a9ead8a95b4d56a69473f896fd4f7a 2023-08-23 10:54:54,454 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/d93a555199be4b358177158493a41499 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/d93a555199be4b358177158493a41499 2023-08-23 10:54:54,455 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/cf2/e96b3fa6fc1e48e2987f1166ba78f070 2023-08-23 10:54:54,457 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62/recovered.edits/85.seqid 2023-08-23 10:54:54,457 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testInvalidSplitKey/c46b502b19bda6c6c44af6dcb42f2a62 2023-08-23 10:54:54,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-08-23 10:54:54,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,461 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-08-23 10:54:54,464 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-08-23 10:54:54,465 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,465 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-08-23 10:54:54,465 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788094465"}]},"ts":"9223372036854775807"} 2023-08-23 10:54:54,467 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-08-23 10:54:54,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c46b502b19bda6c6c44af6dcb42f2a62, NAME => 'testInvalidSplitKey,,1692788092483.c46b502b19bda6c6c44af6dcb42f2a62.', STARTKEY => '', ENDKEY => ''}] 2023-08-23 10:54:54,467 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-08-23 10:54:54,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788094467"}]},"ts":"9223372036854775807"} 2023-08-23 10:54:54,468 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-08-23 10:54:54,470 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:54,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 41 msec 2023-08-23 10:54:54,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-08-23 10:54:54,533 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-08-23 10:54:54,547 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=370 (was 350) - Thread LEAK? -, OpenFileDescriptor=747 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 175), ProcessCount=171 (was 171), AvailableMemoryMB=10821 (was 10831) 2023-08-23 10:54:54,560 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=370, OpenFileDescriptor=747, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=171, AvailableMemoryMB=10820 2023-08-23 10:54:54,560 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:54:54,562 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:54:54,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:54:54,564 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:54,565 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:54:54,566 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:54:54,567 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,568 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 empty. 2023-08-23 10:54:54,568 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,568 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-23 10:54:54,580 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-08-23 10:54:54,581 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => c442a65f48f95816e9ee8b3dc3e4efb1, NAME => 'testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:54:54,588 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:54:54,588 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing c442a65f48f95816e9ee8b3dc3e4efb1, disabling compactions & flushes 2023-08-23 10:54:54,588 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,588 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,588 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. after waiting 0 ms 2023-08-23 10:54:54,589 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,589 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,589 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:54,591 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:54:54,592 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788094592"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788094592"}]},"ts":"1692788094592"} 2023-08-23 10:54:54,593 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:54:54,594 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:54:54,594 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788094594"}]},"ts":"1692788094594"} 2023-08-23 10:54:54,595 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-08-23 10:54:54,598 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:54:54,598 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:54:54,599 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:54:54,599 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:54:54,599 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:54:54,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, ASSIGN}] 2023-08-23 10:54:54,600 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, ASSIGN 2023-08-23 10:54:54,600 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:54:54,751 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:54:54,752 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:54,752 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788094752"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788094752"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788094752"}]},"ts":"1692788094752"} 2023-08-23 10:54:54,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:54:54,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c442a65f48f95816e9ee8b3dc3e4efb1, NAME => 'testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:54:54,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:54:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:54:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,910 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,911 DEBUG [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1 2023-08-23 10:54:54,911 DEBUG [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1 2023-08-23 10:54:54,911 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c442a65f48f95816e9ee8b3dc3e4efb1 columnFamilyName cf1 2023-08-23 10:54:54,912 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] regionserver.HStore(310): Store=c442a65f48f95816e9ee8b3dc3e4efb1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:54:54,912 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,913 DEBUG [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2 2023-08-23 10:54:54,913 DEBUG [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2 2023-08-23 10:54:54,913 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c442a65f48f95816e9ee8b3dc3e4efb1 columnFamilyName cf2 2023-08-23 10:54:54,914 INFO [StoreOpener-c442a65f48f95816e9ee8b3dc3e4efb1-1] regionserver.HStore(310): Store=c442a65f48f95816e9ee8b3dc3e4efb1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:54:54,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:54:54,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:54,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:54:54,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c442a65f48f95816e9ee8b3dc3e4efb1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60285868, jitterRate=-0.10167056322097778}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:54:54,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:54,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1., pid=111, masterSystemTime=1692788094905 2023-08-23 10:54:54,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:54,922 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:54,922 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788094922"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788094922"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788094922"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788094922"}]},"ts":"1692788094922"} 2023-08-23 10:54:54,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-08-23 10:54:54,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 in 169 msec 2023-08-23 10:54:54,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-08-23 10:54:54,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, ASSIGN in 326 msec 2023-08-23 10:54:54,927 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:54:54,927 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788094927"}]},"ts":"1692788094927"} 2023-08-23 10:54:54,928 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-08-23 10:54:54,931 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:54:54,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 368 msec 2023-08-23 10:54:55,068 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,073 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,073 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,073 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,073 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,073 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,073 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,074 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,074 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,074 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,074 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,075 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,075 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,077 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,077 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,079 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:54:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,082 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,089 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/5a33c032e34d4592bd6bd35c7482ce49 2023-08-23 10:54:55,101 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/72bc35a80fd345e4b3e6b9f721a692a2 2023-08-23 10:54:55,104 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/5a33c032e34d4592bd6bd35c7482ce49 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 2023-08-23 10:54:55,108 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:54:55,108 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/72bc35a80fd345e4b3e6b9f721a692a2 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 2023-08-23 10:54:55,112 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:54:55,113 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 34ms, sequenceid=6, compaction requested=false 2023-08-23 10:54:55,114 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:55,114 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,114 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:55,114 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:55,114 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,114 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,114 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,118 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:55,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72e541e3[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:55,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:55,120 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,121 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,124 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:55,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:55,127 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:55,127 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:55,127 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:55,127 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,127 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,131 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,131 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,132 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,133 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,134 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,134 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:55,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,137 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,138 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:55,138 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,138 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,138 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,138 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,238 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:55,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:55,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,251 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,251 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,251 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,251 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,251 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,251 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,252 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,252 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,252 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,252 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,253 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,253 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,255 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,256 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,256 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,256 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,257 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,264 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/d1a224684d074fe59283801ab146e044 2023-08-23 10:54:55,275 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/17912984893b48a392eea32fff350331 2023-08-23 10:54:55,279 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/d1a224684d074fe59283801ab146e044 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 2023-08-23 10:54:55,282 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:54:55,283 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/17912984893b48a392eea32fff350331 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 2023-08-23 10:54:55,287 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:54:55,287 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 31ms, sequenceid=19, compaction requested=false 2023-08-23 10:54:55,288 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:55,288 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,288 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:55,288 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:55,288 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,288 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,288 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,294 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:55,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f99b1e[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:55,296 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:55,296 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:55,297 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:55,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,297 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,299 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,299 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,300 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:55,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:55,329 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:55,329 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:55,329 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:55,329 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,329 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,333 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,334 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,335 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,336 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,340 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:55,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:55,341 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,342 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,442 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:55,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:55,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:55,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,454 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,454 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,455 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,455 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,455 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,455 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,456 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,456 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,457 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,457 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,458 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,459 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,459 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,459 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,459 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:55,460 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,462 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,462 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,462 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,462 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,462 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,475 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 2023-08-23 10:54:55,487 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/a9f142bcc2064dc19f5472f21e945e66 2023-08-23 10:54:55,490 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 2023-08-23 10:54:55,494 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:54:55,494 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/a9f142bcc2064dc19f5472f21e945e66 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 2023-08-23 10:54:55,497 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:54:55,498 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 39ms, sequenceid=32, compaction requested=true 2023-08-23 10:54:55,498 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:55,498 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,498 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:55,498 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:55,498 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,498 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,498 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,500 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,500 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,500 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,500 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:55,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@288ade2b[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:55,502 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:55,502 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,503 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:55,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,503 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,504 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,505 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,507 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,507 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,507 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,507 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:55,510 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:55,510 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:55,510 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:55,510 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:55,510 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,510 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,511 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,512 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,513 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,513 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,514 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,515 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,515 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,519 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,519 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,519 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,519 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,520 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,520 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,521 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,621 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:55,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:55,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:55,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,634 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,634 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,636 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,636 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,636 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,636 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,637 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,637 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,640 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,640 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,641 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:55,650 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/0c7690172e99476d9731417f884112c9 2023-08-23 10:54:55,660 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/af3523292fe844e591c93bb7bdb3a2aa 2023-08-23 10:54:55,664 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/0c7690172e99476d9731417f884112c9 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 2023-08-23 10:54:55,668 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:54:55,669 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/af3523292fe844e591c93bb7bdb3a2aa as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa 2023-08-23 10:54:55,672 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:54:55,673 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 33ms, sequenceid=45, compaction requested=true 2023-08-23 10:54:55,673 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:55,673 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:55,674 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,674 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,674 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,675 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,676 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:55,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7055b8a7[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:55,677 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:55,677 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,678 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,678 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,678 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:55,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:55,681 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,681 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,681 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,681 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,682 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,682 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,682 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,682 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,682 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,682 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:55,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:55,686 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:55,686 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:55,686 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:55,686 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,686 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,688 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,687 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,688 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,689 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,690 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,691 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,691 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,697 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:55,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:55,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,697 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,798 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:55,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:55,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:55,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,811 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,811 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,811 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,811 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,811 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,811 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,813 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,813 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,813 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,814 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,817 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,817 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,817 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:55,818 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,818 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,818 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,827 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/289ea5d138214f1892651e8a72a0c36a 2023-08-23 10:54:55,836 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/5d91a7eee88048908d1af192d24c1ebb 2023-08-23 10:54:55,842 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/289ea5d138214f1892651e8a72a0c36a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a 2023-08-23 10:54:55,845 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:54:55,846 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/5d91a7eee88048908d1af192d24c1ebb as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb 2023-08-23 10:54:55,850 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:54:55,851 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 34ms, sequenceid=58, compaction requested=true 2023-08-23 10:54:55,851 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:55,851 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,851 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:55,851 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:55,851 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,851 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,851 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28e70517[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:55,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:55,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,860 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,860 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,861 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:55,860 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,860 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,862 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:55,866 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:55,866 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:55,866 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:55,866 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,866 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,869 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,870 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:55,870 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:55,871 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,875 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,875 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,875 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,875 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:55,876 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,875 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:55,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,876 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:55,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,877 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:55,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:55,976 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:55,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:55,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:54:55,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-08-23 10:54:55,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-08-23 10:54:55,991 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:55,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-08-23 10:54:55,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-08-23 10:54:55,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,992 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,993 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,992 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:55,993 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:55,993 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,994 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,994 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,994 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,994 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,994 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:54:55,995 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:54:55,995 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:55,995 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. started... 2023-08-23 10:54:55,995 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:54:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:56,007 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/16f697e517254d14b1f5ad39fe37d855 2023-08-23 10:54:56,018 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/c7bca92157974ef9ad86ff6fc5d47297 2023-08-23 10:54:56,021 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/16f697e517254d14b1f5ad39fe37d855 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 2023-08-23 10:54:56,025 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:54:56,026 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/c7bca92157974ef9ad86ff6fc5d47297 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 2023-08-23 10:54:56,028 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:54:56,029 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 34ms, sequenceid=71, compaction requested=true 2023-08-23 10:54:56,029 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:56,029 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:56,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:54:56,029 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:54:56,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-08-23 10:54:56,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-08-23 10:54:56,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:56,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:56,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,037 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:54:56,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:56,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:56,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:56,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-08-23 10:54:56,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@658b4a7e[Count = 0] remaining members to acquire global barrier 2023-08-23 10:54:56,039 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-08-23 10:54:56,039 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,040 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,040 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,040 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:56,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:56,041 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,041 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:54:56,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-08-23 10:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:54:56,042 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:56,042 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:56,042 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,042 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:56,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:56,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:56,043 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-08-23 10:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:56,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-08-23 10:54:56,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:56,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-08-23 10:54:56,047 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-08-23 10:54:56,047 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:54:56,047 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:54:56,047 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-08-23 10:54:56,047 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:54:56,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,049 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,050 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,050 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,052 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:54:56,052 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-08-23 10:54:56,052 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,057 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,057 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:56,057 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:56,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:56,058 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:56,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:56,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-08-23 10:54:56,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:54:56,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:54:56,058 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:54:56,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:54:56,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:54:56,059 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,059 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:54:56,059 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,059 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,059 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-08-23 10:54:56,159 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:54:56,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:54:56,417 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-08-23 10:54:56,418 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:56,419 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,421 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:56,421 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:56,421 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:56,670 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:56,670 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:56,671 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:56,672 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-08-23 10:54:56,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:56,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:56,673 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,674 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,675 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:56,676 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,677 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,677 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,677 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,677 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:56,677 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,678 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:56,678 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,678 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,679 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,679 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-23 10:54:56,679 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:56,680 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:56,680 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788126680 2023-08-23 10:54:56,680 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:56,682 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,685 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:56,685 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:56,685 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:56,685 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788156685 2023-08-23 10:54:56,685 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788216685 2023-08-23 10:54:56,686 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-08-23 10:54:56,686 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:56,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:56,688 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:56,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:56,936 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:56,936 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:56,937 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:56,938 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,939 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,940 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:56,941 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,941 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:56,941 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:56,941 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,941 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,942 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,943 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,944 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:56,944 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:56,944 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,944 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:56,944 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,944 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,944 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,944 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-23 10:54:56,945 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:56,945 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:56,946 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788126946 2023-08-23 10:54:56,947 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,947 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:56,949 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:56,949 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:56,950 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:56,950 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788156950 2023-08-23 10:54:56,950 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788216950 2023-08-23 10:54:56,950 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:54:56,950 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:56,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN}] 2023-08-23 10:54:56,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:56,952 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:56,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:56,952 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:57,200 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,200 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:57,200 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:57,200 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:57,200 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:57,200 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:57,201 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:57,201 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:57,201 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:57,201 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:57,203 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-08-23 10:54:57,203 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:57,204 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:57,204 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:57,204 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:57,204 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,205 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:57,205 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,205 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,205 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,205 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,206 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,207 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,207 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:57,207 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,207 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,207 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,208 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:57,208 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,208 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,208 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,208 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,209 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:57,209 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,209 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,209 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,210 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,210 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,210 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:57,210 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,210 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:57,210 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,210 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,210 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,211 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-23 10:54:57,211 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:57,212 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:57,213 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788127213 2023-08-23 10:54:57,214 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,214 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:57,217 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:57,217 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:57,217 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:57,217 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788157217 2023-08-23 10:54:57,217 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788217217 2023-08-23 10:54:57,217 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:54:57,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN}] 2023-08-23 10:54:57,218 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:57,219 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:57,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:57,220 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788097220"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788097220"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788097220"}]},"ts":"1692788097220"} 2023-08-23 10:54:57,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:54:57,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c442a65f48f95816e9ee8b3dc3e4efb1, disabling compactions & flushes 2023-08-23 10:54:57,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:57,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:57,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. after waiting 0 ms 2023-08-23 10:54:57,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:57,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing c442a65f48f95816e9ee8b3dc3e4efb1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:54:57,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/d5a1904b4fd4481384845a0eb8653a35 2023-08-23 10:54:57,394 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/e8383895f66145189e9852ca0c3de1d7 2023-08-23 10:54:57,397 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf1/d5a1904b4fd4481384845a0eb8653a35 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 2023-08-23 10:54:57,401 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:54:57,402 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/.tmp/cf2/e8383895f66145189e9852ca0c3de1d7 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 2023-08-23 10:54:57,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:54:57,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c442a65f48f95816e9ee8b3dc3e4efb1 in 32ms, sequenceid=82, compaction requested=true 2023-08-23 10:54:57,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:54:57,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:54:57,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. 2023-08-23 10:54:57,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c442a65f48f95816e9ee8b3dc3e4efb1: 2023-08-23 10:54:57,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED 2023-08-23 10:54:57,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788097413"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788097413"}]},"ts":"1692788097413"} 2023-08-23 10:54:57,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-08-23 10:54:57,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 in 193 msec 2023-08-23 10:54:57,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-08-23 10:54:57,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN in 199 msec 2023-08-23 10:54:57,419 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:57,419 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:57,419 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:57,467 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:57,468 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:57,468 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:57,469 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:57,470 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-08-23 10:54:57,470 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:57,470 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:57,470 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:57,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:57,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:57,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:57,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,471 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:57,472 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,472 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:57,472 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,472 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:57,473 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,473 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:57,473 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,473 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:57,474 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:57,474 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,474 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:57,474 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,474 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:57,475 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,475 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:57,475 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,475 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:57,475 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,476 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:57,476 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,476 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,476 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:57,477 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,477 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:57,477 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,477 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,477 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:57,478 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,478 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,478 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:57,478 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:57,478 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-23 10:54:57,479 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:57,479 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:57,480 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788127480 2023-08-23 10:54:57,480 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:57,481 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:57,483 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:57,484 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:57,484 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:57,484 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788157484 2023-08-23 10:54:57,484 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788217484 2023-08-23 10:54:57,484 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:54:57,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:57,495 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=c442a65f48f95816e9ee8b3dc3e4efb1, threads=14 2023-08-23 10:54:57,495 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,495 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,495 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,495 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,499 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,500 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,500 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,500 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,500 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,501 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,501 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,501 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,502 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,503 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,668 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,684 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,695 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,705 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,705 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,705 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,705 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,707 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,708 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,708 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,708 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,709 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:57,746 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:54:57,956 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,043 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,043 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region c442a65f48f95816e9ee8b3dc3e4efb1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:54:58,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:58,072 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:58,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:58,234 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,234 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:58,235 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:58,235 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:58,236 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:58,238 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-08-23 10:54:58,238 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:58,238 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:58,238 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:58,239 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:58,239 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:58,239 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:58,239 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,240 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:58,240 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,240 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,240 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,240 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:58,241 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,241 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,241 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,241 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:58,242 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:58,242 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,242 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,242 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,242 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,243 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,244 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,244 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,244 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:58,244 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,245 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,245 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,245 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,245 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,245 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:58,246 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,246 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:58,246 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:58,246 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:58,246 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,246 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,246 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,247 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:58,247 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:58,247 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-08-23 10:54:58,247 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:58,248 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:58,248 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788128248 2023-08-23 10:54:58,249 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:58,250 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,252 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:58,253 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:58,253 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:58,253 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788158253 2023-08-23 10:54:58,253 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788218253 2023-08-23 10:54:58,253 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:54:58,261 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=c442a65f48f95816e9ee8b3dc3e4efb1, threads=14 2023-08-23 10:54:58,261 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,261 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,261 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,264 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,264 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,265 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,269 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,269 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,269 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,413 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,430 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,442 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,448 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,450 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,450 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,451 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,453 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,455 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,456 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,457 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,469 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,469 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,471 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 for region: c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:58,471 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region c442a65f48f95816e9ee8b3dc3e4efb1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:54:58,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:58,508 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:54:58,509 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:54:58,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:58,509 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:58,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:58,754 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:58,754 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:58,755 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:58,755 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:58,756 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-08-23 10:54:58,756 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:58,756 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,757 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,758 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,759 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:58,760 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:58,761 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:58,761 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,761 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,761 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,762 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:58,762 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:58,762 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-23 10:54:58,762 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:58,763 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:58,763 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788128763 2023-08-23 10:54:58,765 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,765 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:58,767 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:58,767 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:58,767 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:58,767 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788158767 2023-08-23 10:54:58,768 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788218768 2023-08-23 10:54:58,768 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:54:58,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:58,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:58,772 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:58,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:58,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:58,937 ERROR [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 0db35eca-a409-482d-977c-53ac02140840) service to localhost/127.0.0.1:34277 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:59,018 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:59,018 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:59,019 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:59,019 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:59,020 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-08-23 10:54:59,020 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:59,020 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:59,021 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:59,022 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,023 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,024 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,024 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,024 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:59,024 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,025 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,025 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,025 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,025 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,026 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:59,026 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,026 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:59,026 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,026 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:59,026 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,026 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,027 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,027 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,027 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,027 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-08-23 10:54:59,027 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:59,028 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:59,028 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788129028 2023-08-23 10:54:59,030 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,030 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:59,032 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:59,032 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:59,032 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:59,032 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788159032 2023-08-23 10:54:59,032 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788219032 2023-08-23 10:54:59,032 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:54:59,032 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:59,034 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788099034"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1692788099034"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1692788099034"}]},"ts":"1692788099034"} 2023-08-23 10:54:59,035 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099034"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788099034"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788099034"}]},"ts":"1692788099034"} 2023-08-23 10:54:59,035 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099034"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788099034"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788099034"}]},"ts":"1692788099034"} 2023-08-23 10:54:59,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:59,036 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:59,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:59,283 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:59,283 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:59,284 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:59,285 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-08-23 10:54:59,285 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:59,285 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:59,286 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:59,286 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:59,286 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:59,286 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:59,287 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,287 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:59,287 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,287 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,287 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,288 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:59,288 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,288 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,288 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,289 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:59,289 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:59,289 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,289 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,289 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:59,290 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,291 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,291 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,291 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,292 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,293 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:59,293 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,293 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:59,293 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,293 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:59,294 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,294 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,294 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,294 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,294 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,294 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-08-23 10:54:59,295 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:59,295 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:59,295 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788129295 2023-08-23 10:54:59,298 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:59,298 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,300 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:59,300 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:59,300 INFO [Listener at localhost/34585] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532, table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:59,300 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=f904e288193db0dbb00e8e9fc48e0c7f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,300 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=25ba0c6a926dacd93f1b3db70486e8fa, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,300 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:59,300 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788159300 2023-08-23 10:54:59,300 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788219300 2023-08-23 10:54:59,300 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-08-23 10:54:59,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1692788099301"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1692788099301"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1692788099301"}]},"ts":"1692788099301"} 2023-08-23 10:54:59,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099301"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788099301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788099301"}]},"ts":"1692788099301"} 2023-08-23 10:54:59,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099301"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788099301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788099301"}]},"ts":"1692788099301"} 2023-08-23 10:54:59,302 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:59,304 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:59,304 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:59,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:59,551 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:59,551 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:59,552 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:59,552 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:59,552 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:59,553 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:59,554 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,555 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,556 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,557 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,557 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:59,557 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,557 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,558 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,558 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:59,558 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,558 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,558 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,559 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,559 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:59,559 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,559 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,560 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,560 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,560 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,560 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:59,561 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,561 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,561 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,561 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,562 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-08-23 10:54:59,562 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:59,563 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:59,563 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788129563 2023-08-23 10:54:59,565 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,565 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:59,568 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:59,568 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:59,568 INFO [Listener at localhost/34585] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532, table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:59,568 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=f904e288193db0dbb00e8e9fc48e0c7f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,568 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=25ba0c6a926dacd93f1b3db70486e8fa, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,568 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:59,568 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788159568 2023-08-23 10:54:59,568 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788219568 2023-08-23 10:54:59,569 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:54:59,569 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:59,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN}] 2023-08-23 10:54:59,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:54:59,570 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:54:59,571 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:54:59,819 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:54:59,819 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:54:59,820 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:54:59,821 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-08-23 10:54:59,821 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:54:59,821 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:54:59,821 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,822 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:54:59,823 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,824 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,825 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:54:59,826 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,826 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,826 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,826 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:54:59,827 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-08-23 10:54:59,827 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:54:59,827 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:54:59,828 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788129828 2023-08-23 10:54:59,829 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:54:59,830 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:54:59,832 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:54:59,832 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:54:59,832 INFO [Listener at localhost/34585] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532, table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:54:59,832 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=f904e288193db0dbb00e8e9fc48e0c7f, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,833 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=25ba0c6a926dacd93f1b3db70486e8fa, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-08-23 10:54:59,833 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:54:59,833 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788159833 2023-08-23 10:54:59,833 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788219833 2023-08-23 10:54:59,833 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-08-23 10:54:59,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN}] 2023-08-23 10:54:59,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:54:59,835 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN 2023-08-23 10:54:59,836 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:54:59,836 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN 2023-08-23 10:54:59,837 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:54:59,986 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:54:59,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f904e288193db0dbb00e8e9fc48e0c7f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099988"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788099988"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788099988"}]},"ts":"1692788099988"} 2023-08-23 10:54:59,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:54:59,992 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=25ba0c6a926dacd93f1b3db70486e8fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:54:59,992 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788099992"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788099992"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788099992"}]},"ts":"1692788099992"} 2023-08-23 10:54:59,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:00,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 25ba0c6a926dacd93f1b3db70486e8fa, NAME => 'testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.', STARTKEY => '41', ENDKEY => ''} 2023-08-23 10:55:00,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:00,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:00,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,145 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,146 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1 2023-08-23 10:55:00,146 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1 2023-08-23 10:55:00,146 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25ba0c6a926dacd93f1b3db70486e8fa columnFamilyName cf1 2023-08-23 10:55:00,159 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-top 2023-08-23 10:55:00,163 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-top 2023-08-23 10:55:00,166 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-top 2023-08-23 10:55:00,169 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-top 2023-08-23 10:55:00,173 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-top 2023-08-23 10:55:00,176 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-top 2023-08-23 10:55:00,179 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-top 2023-08-23 10:55:00,179 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(310): Store=25ba0c6a926dacd93f1b3db70486e8fa/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:00,180 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,180 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2 2023-08-23 10:55:00,180 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2 2023-08-23 10:55:00,181 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 25ba0c6a926dacd93f1b3db70486e8fa columnFamilyName cf2 2023-08-23 10:55:00,193 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-top 2023-08-23 10:55:00,196 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-top 2023-08-23 10:55:00,200 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-top 2023-08-23 10:55:00,204 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-top 2023-08-23 10:55:00,208 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-top 2023-08-23 10:55:00,211 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-top 2023-08-23 10:55:00,214 DEBUG [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-top 2023-08-23 10:55:00,214 INFO [StoreOpener-25ba0c6a926dacd93f1b3db70486e8fa-1] regionserver.HStore(310): Store=25ba0c6a926dacd93f1b3db70486e8fa/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:00,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:00,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 25ba0c6a926dacd93f1b3db70486e8fa; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60838137, jitterRate=-0.09344111382961273}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:00,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 25ba0c6a926dacd93f1b3db70486e8fa: 2023-08-23 10:55:00,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa., pid=118, masterSystemTime=1692788100141 2023-08-23 10:55:00,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:55:00,221 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:00,221 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:00,221 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,221 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): 25ba0c6a926dacd93f1b3db70486e8fa/cf1 is initiating minor compaction (all files) 2023-08-23 10:55:00,221 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,221 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): 25ba0c6a926dacd93f1b3db70486e8fa/cf2 is initiating minor compaction (all files) 2023-08-23 10:55:00,221 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 25ba0c6a926dacd93f1b3db70486e8fa/cf1 in testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,221 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 25ba0c6a926dacd93f1b3db70486e8fa/cf2 in testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,222 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/.tmp, totalSize=34.9 K 2023-08-23 10:55:00,222 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/.tmp, totalSize=34.9 K 2023-08-23 10:55:00,222 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788095066 2023-08-23 10:55:00,222 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788095066 2023-08-23 10:55:00,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:00,223 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788095240 2023-08-23 10:55:00,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,223 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788095240 2023-08-23 10:55:00,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f904e288193db0dbb00e8e9fc48e0c7f, NAME => 'testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.', STARTKEY => '', ENDKEY => '41'} 2023-08-23 10:55:00,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=25ba0c6a926dacd93f1b3db70486e8fa, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,223 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788095443 2023-08-23 10:55:00,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:00,223 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788095443 2023-08-23 10:55:00,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788100223"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788100223"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788100223"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788100223"}]},"ts":"1692788100223"} 2023-08-23 10:55:00,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:00,223 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788095623 2023-08-23 10:55:00,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,224 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788095623 2023-08-23 10:55:00,224 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788095799 2023-08-23 10:55:00,224 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788095978 2023-08-23 10:55:00,224 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788095799 2023-08-23 10:55:00,225 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788096160 2023-08-23 10:55:00,225 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788095978 2023-08-23 10:55:00,225 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788096160 2023-08-23 10:55:00,226 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,227 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1 2023-08-23 10:55:00,227 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1 2023-08-23 10:55:00,227 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f904e288193db0dbb00e8e9fc48e0c7f columnFamilyName cf1 2023-08-23 10:55:00,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-08-23 10:55:00,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532 in 232 msec 2023-08-23 10:55:00,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN in 396 msec 2023-08-23 10:55:00,241 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-bottom 2023-08-23 10:55:00,249 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): 25ba0c6a926dacd93f1b3db70486e8fa#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:00,249 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): 25ba0c6a926dacd93f1b3db70486e8fa#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:00,253 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-bottom 2023-08-23 10:55:00,262 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-bottom 2023-08-23 10:55:00,270 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-bottom 2023-08-23 10:55:00,275 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/.tmp/cf1/5c07b43e4df14c1688aac8e2e1296745 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5c07b43e4df14c1688aac8e2e1296745 2023-08-23 10:55:00,275 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-bottom 2023-08-23 10:55:00,279 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/.tmp/cf2/77f732a81ccf4e9788c073c32f36c0de as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/77f732a81ccf4e9788c073c32f36c0de 2023-08-23 10:55:00,282 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-bottom 2023-08-23 10:55:00,282 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 25ba0c6a926dacd93f1b3db70486e8fa/cf1 of 25ba0c6a926dacd93f1b3db70486e8fa into 5c07b43e4df14c1688aac8e2e1296745(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:00,283 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 25ba0c6a926dacd93f1b3db70486e8fa: 2023-08-23 10:55:00,283 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa., storeName=25ba0c6a926dacd93f1b3db70486e8fa/cf1, priority=9, startTime=1692788100220; duration=0sec 2023-08-23 10:55:00,283 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,284 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 25ba0c6a926dacd93f1b3db70486e8fa/cf2 of 25ba0c6a926dacd93f1b3db70486e8fa into 77f732a81ccf4e9788c073c32f36c0de(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:00,284 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 25ba0c6a926dacd93f1b3db70486e8fa: 2023-08-23 10:55:00,284 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa., storeName=25ba0c6a926dacd93f1b3db70486e8fa/cf2, priority=9, startTime=1692788100221; duration=0sec 2023-08-23 10:55:00,284 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,287 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-bottom 2023-08-23 10:55:00,287 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(310): Store=f904e288193db0dbb00e8e9fc48e0c7f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:00,287 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,288 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2 2023-08-23 10:55:00,288 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2 2023-08-23 10:55:00,288 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f904e288193db0dbb00e8e9fc48e0c7f columnFamilyName cf2 2023-08-23 10:55:00,300 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-bottom 2023-08-23 10:55:00,304 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-bottom 2023-08-23 10:55:00,307 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-bottom 2023-08-23 10:55:00,310 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-bottom 2023-08-23 10:55:00,313 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-bottom 2023-08-23 10:55:00,318 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-bottom 2023-08-23 10:55:00,322 DEBUG [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-bottom 2023-08-23 10:55:00,322 INFO [StoreOpener-f904e288193db0dbb00e8e9fc48e0c7f-1] regionserver.HStore(310): Store=f904e288193db0dbb00e8e9fc48e0c7f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:00,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:00,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:00,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f904e288193db0dbb00e8e9fc48e0c7f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61237153, jitterRate=-0.08749531209468842}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:00,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f904e288193db0dbb00e8e9fc48e0c7f: 2023-08-23 10:55:00,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f., pid=117, masterSystemTime=1692788100141 2023-08-23 10:55:00,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:55:00,328 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:00,328 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:00,328 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,328 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): f904e288193db0dbb00e8e9fc48e0c7f/cf1 is initiating minor compaction (all files) 2023-08-23 10:55:00,328 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,329 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f904e288193db0dbb00e8e9fc48e0c7f/cf1 in testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,329 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): f904e288193db0dbb00e8e9fc48e0c7f/cf2 is initiating minor compaction (all files) 2023-08-23 10:55:00,329 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f904e288193db0dbb00e8e9fc48e0c7f/cf2 in testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,329 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/.tmp, totalSize=34.9 K 2023-08-23 10:55:00,329 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/.tmp, totalSize=34.9 K 2023-08-23 10:55:00,330 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788095066 2023-08-23 10:55:00,330 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788095066 2023-08-23 10:55:00,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:00,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f904e288193db0dbb00e8e9fc48e0c7f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788100330"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788100330"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788100330"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788100330"}]},"ts":"1692788100330"} 2023-08-23 10:55:00,331 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788095240 2023-08-23 10:55:00,331 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788095240 2023-08-23 10:55:00,332 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788095443 2023-08-23 10:55:00,332 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788095443 2023-08-23 10:55:00,332 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788095623 2023-08-23 10:55:00,332 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788095623 2023-08-23 10:55:00,332 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788095799 2023-08-23 10:55:00,333 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788095799 2023-08-23 10:55:00,333 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788095978 2023-08-23 10:55:00,333 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788095978 2023-08-23 10:55:00,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-08-23 10:55:00,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532 in 343 msec 2023-08-23 10:55:00,334 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788096160 2023-08-23 10:55:00,334 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788096160 2023-08-23 10:55:00,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-08-23 10:55:00,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN in 501 msec 2023-08-23 10:55:00,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-08-23 10:55:00,335 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:55:00,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-08-23 10:55:00,343 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): f904e288193db0dbb00e8e9fc48e0c7f#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:00,344 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): f904e288193db0dbb00e8e9fc48e0c7f#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:00,363 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/.tmp/cf2/7bd40d0366c6404fa9daddd454f9ef6c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/7bd40d0366c6404fa9daddd454f9ef6c 2023-08-23 10:55:00,363 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/.tmp/cf1/07fac9b62338458bae306b09bc51bac4 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/07fac9b62338458bae306b09bc51bac4 2023-08-23 10:55:00,367 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f904e288193db0dbb00e8e9fc48e0c7f/cf2 of f904e288193db0dbb00e8e9fc48e0c7f into 7bd40d0366c6404fa9daddd454f9ef6c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:00,367 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f904e288193db0dbb00e8e9fc48e0c7f: 2023-08-23 10:55:00,367 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f., storeName=f904e288193db0dbb00e8e9fc48e0c7f/cf2, priority=9, startTime=1692788100328; duration=0sec 2023-08-23 10:55:00,368 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,372 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f904e288193db0dbb00e8e9fc48e0c7f/cf1 of f904e288193db0dbb00e8e9fc48e0c7f into 07fac9b62338458bae306b09bc51bac4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:00,372 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f904e288193db0dbb00e8e9fc48e0c7f: 2023-08-23 10:55:00,372 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f., storeName=f904e288193db0dbb00e8e9fc48e0c7f/cf1, priority=9, startTime=1692788100327; duration=0sec 2023-08-23 10:55:00,372 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:00,583 INFO [Listener at localhost/34585] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,583 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-08-23 10:55:00,583 INFO [Listener at localhost/34585] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:55:00,584 INFO [Listener at localhost/34585] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:55:00,584 ERROR [Listener at localhost/34585] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-08-23 10:55:00,584 INFO [Listener at localhost/34585] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-08-23 10:55:00,584 INFO [Listener at localhost/34585] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-08-23 10:55:00,584 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-08-23 10:55:00,584 INFO [Listener at localhost/34585] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-08-23 10:55:00,585 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-08-23 10:55:00,586 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-08-23 10:55:00,587 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-08-23 10:55:00,588 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-08-23 10:55:00,588 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-08-23 10:55:00,588 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-08-23 10:55:00,588 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:55:00,588 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=83b16afbbfa1198d46801c8c70153e1b, daughterA=f1547d5000cdc5a6e9fd38083a69821a, daughterB=196b1478f733e5db2758765b9cd2bb43 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4240699d79069c083bb040ccf4f51d95, daughterA=d8f6aa47800ed8299ad4e206a840de11, daughterB=eddadea6d5040903b4cbcb5f719e6ffb 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-08-23 10:55:00,589 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-08-23 10:55:00,590 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:55:00,590 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=84b37956ab3c45570fda115a32fc8299, daughterA=20b4d239aacb53b0c1fb250029e2b764, daughterB=97581af8a5a64d4ea1fc42cf27d9b58d 2023-08-23 10:55:00,590 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=84b37956ab3c45570fda115a32fc8299, ASSIGN 2023-08-23 10:55:00,590 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:55:00,590 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=1a4eca4d5307f5febb75fe13c4379735, daughterA=ffedba6c94ad43305ebb2566148be64d, daughterB=88c3108086fd6ceee8569a37c2048096 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:55:00,591 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=8cd300f31326ee729e983413d0ee732f, daughterA=e2d75dd262274c53186d4a8a3dc32e21, daughterB=0c8db88bd15dc03c296a5c5fe5edfc04 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37675,1692787997438, table=testRollbackForSplitTableRegionWithReplica, region=fa325bce8f770acd7b48d1ba76647d7b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=52c62d6f8ed74063b747c3f1cbbc4ccf, daughterA=24f70ae61dbb382ae20ce00747d5e751, daughterB=43e1e10ed6371e15da29e7c786dfa1c2 2023-08-23 10:55:00,592 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:55:00,593 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-08-23 10:55:00,593 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-08-23 10:55:00,593 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-08-23 10:55:00,593 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-08-23 10:55:00,593 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,595 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa held the lock before restarting, call acquireLock to restore it. 2023-08-23 10:55:00,596 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,596 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,596 INFO [Listener at localhost/34585] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure c442a65f48f95816e9ee8b3dc3e4efb1, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,596 DEBUG [Listener at localhost/34585] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532 didn't hold the lock before restarting, skip acquiring lock. 2023-08-23 10:55:00,597 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-08-23 10:55:00,597 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-08-23 10:55:00,598 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41821,1692787997489, table=hbase:meta, region=1588230740 2023-08-23 10:55:00,598 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1692788130598 2023-08-23 10:55:00,600 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:00,600 DEBUG [Listener at localhost/34585] assignment.AssignmentManager(1517): Joining cluster... 2023-08-23 10:55:00,603 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=03da722d073e4770f60de66574c27a90, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37675,1692787997438, regionLocation=jenkins-hbase4.apache.org,37675,1692787997438, openSeqNum=2 2023-08-23 10:55:00,603 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=c442a65f48f95816e9ee8b3dc3e4efb1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=2 2023-08-23 10:55:00,603 INFO [Listener at localhost/34585] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43499,1692787997532, table=testRecoveryAndDoubleExecution, region=c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:00,603 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=f904e288193db0dbb00e8e9fc48e0c7f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=86 2023-08-23 10:55:00,604 DEBUG [Listener at localhost/34585] assignment.RegionStateStore(147): Load hbase:meta entry region=25ba0c6a926dacd93f1b3db70486e8fa, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43499,1692787997532, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532, openSeqNum=86 2023-08-23 10:55:00,604 INFO [Listener at localhost/34585] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-08-23 10:55:00,604 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1692788160604 2023-08-23 10:55:00,604 INFO [Listener at localhost/34585] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1692788220604 2023-08-23 10:55:00,604 INFO [Listener at localhost/34585] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-08-23 10:55:00,604 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-08-23 10:55:00,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=c442a65f48f95816e9ee8b3dc3e4efb1, daughterA=f904e288193db0dbb00e8e9fc48e0c7f, daughterB=25ba0c6a926dacd93f1b3db70486e8fa in 4.1850 sec 2023-08-23 10:55:00,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43499] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:38534 deadline: 1692788160855, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1. is not online on jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,967 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:55:00,974 INFO [Listener at localhost/34585] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-23 10:55:00,975 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:55:00,985 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:55:00,986 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-08-23 10:55:00,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-08-23 10:55:00,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:00,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-23 10:55:00,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788100990"}]},"ts":"1692788100990"} 2023-08-23 10:55:00,991 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-08-23 10:55:00,992 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-08-23 10:55:00,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, UNASSIGN}] 2023-08-23 10:55:00,993 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, UNASSIGN 2023-08-23 10:55:00,994 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=25ba0c6a926dacd93f1b3db70486e8fa, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,994 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788100994"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788100994"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788100994"}]},"ts":"1692788100994"} 2023-08-23 10:55:00,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:00,997 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, UNASSIGN 2023-08-23 10:55:00,997 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f904e288193db0dbb00e8e9fc48e0c7f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:00,997 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788100997"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788100997"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788100997"}]},"ts":"1692788100997"} 2023-08-23 10:55:00,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:01,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-23 10:55:01,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:01,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f904e288193db0dbb00e8e9fc48e0c7f, disabling compactions & flushes 2023-08-23 10:55:01,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:01,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:01,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. after waiting 0 ms 2023-08-23 10:55:01,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:01,147 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-bottom] to archive 2023-08-23 10:55:01,148 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:01,150 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,151 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,152 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,153 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,154 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,155 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,156 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,157 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-bottom] to archive 2023-08-23 10:55:01,158 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:01,159 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,160 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,161 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,162 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,163 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,164 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,165 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:55:01,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:01,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f. 2023-08-23 10:55:01,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f904e288193db0dbb00e8e9fc48e0c7f: 2023-08-23 10:55:01,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:01,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:01,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 25ba0c6a926dacd93f1b3db70486e8fa, disabling compactions & flushes 2023-08-23 10:55:01,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:01,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:01,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. after waiting 0 ms 2023-08-23 10:55:01,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:01,172 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f904e288193db0dbb00e8e9fc48e0c7f, regionState=CLOSED 2023-08-23 10:55:01,172 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788101172"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788101172"}]},"ts":"1692788101172"} 2023-08-23 10:55:01,173 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35-top] to archive 2023-08-23 10:55:01,174 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:01,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-08-23 10:55:01,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure f904e288193db0dbb00e8e9fc48e0c7f, server=jenkins-hbase4.apache.org,43499,1692787997532 in 175 msec 2023-08-23 10:55:01,175 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f904e288193db0dbb00e8e9fc48e0c7f, UNASSIGN in 182 msec 2023-08-23 10:55:01,176 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,177 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,178 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,179 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,180 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,181 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,182 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7-top] to archive 2023-08-23 10:55:01,183 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:01,184 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,185 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,186 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,187 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,188 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,189 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,190 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:55:01,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:01,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa. 2023-08-23 10:55:01,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 25ba0c6a926dacd93f1b3db70486e8fa: 2023-08-23 10:55:01,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:01,197 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=25ba0c6a926dacd93f1b3db70486e8fa, regionState=CLOSED 2023-08-23 10:55:01,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1692788101197"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788101197"}]},"ts":"1692788101197"} 2023-08-23 10:55:01,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-08-23 10:55:01,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 25ba0c6a926dacd93f1b3db70486e8fa, server=jenkins-hbase4.apache.org,43499,1692787997532 in 203 msec 2023-08-23 10:55:01,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-08-23 10:55:01,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=25ba0c6a926dacd93f1b3db70486e8fa, UNASSIGN in 207 msec 2023-08-23 10:55:01,202 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788101202"}]},"ts":"1692788101202"} 2023-08-23 10:55:01,203 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-08-23 10:55:01,204 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-08-23 10:55:01,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 218 msec 2023-08-23 10:55:01,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-08-23 10:55:01,292 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-08-23 10:55:01,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-08-23 10:55:01,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,295 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,296 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-23 10:55:01,300 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,300 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:01,300 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:01,302 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/recovered.edits] 2023-08-23 10:55:01,303 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/recovered.edits] 2023-08-23 10:55:01,303 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/recovered.edits] 2023-08-23 10:55:01,306 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/07fac9b62338458bae306b09bc51bac4 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf1/07fac9b62338458bae306b09bc51bac4 2023-08-23 10:55:01,306 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/.regioninfo to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/.regioninfo 2023-08-23 10:55:01,307 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/0c7690172e99476d9731417f884112c9 2023-08-23 10:55:01,309 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/16f697e517254d14b1f5ad39fe37d855 2023-08-23 10:55:01,309 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/7bd40d0366c6404fa9daddd454f9ef6c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/cf2/7bd40d0366c6404fa9daddd454f9ef6c 2023-08-23 10:55:01,310 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,311 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/289ea5d138214f1892651e8a72a0c36a 2023-08-23 10:55:01,311 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,312 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/5a33c032e34d4592bd6bd35c7482ce49 2023-08-23 10:55:01,312 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/.regioninfo to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/.regioninfo 2023-08-23 10:55:01,313 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,313 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382 2023-08-23 10:55:01,315 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,315 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d1a224684d074fe59283801ab146e044 2023-08-23 10:55:01,315 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/0c7690172e99476d9731417f884112c9.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,316 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,317 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf1/d5a1904b4fd4481384845a0eb8653a35 2023-08-23 10:55:01,317 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/16f697e517254d14b1f5ad39fe37d855.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,318 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,318 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/289ea5d138214f1892651e8a72a0c36a.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,319 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,320 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/5a33c032e34d4592bd6bd35c7482ce49.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,320 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/17912984893b48a392eea32fff350331 2023-08-23 10:55:01,321 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/ae3d6ae5cc504ab6b77ae41e8ebd9382.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,322 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/5d91a7eee88048908d1af192d24c1ebb 2023-08-23 10:55:01,322 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,323 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d1a224684d074fe59283801ab146e044.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,323 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/72bc35a80fd345e4b3e6b9f721a692a2 2023-08-23 10:55:01,323 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,324 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf1/d5a1904b4fd4481384845a0eb8653a35.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,324 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/a9f142bcc2064dc19f5472f21e945e66 2023-08-23 10:55:01,324 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,325 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/af3523292fe844e591c93bb7bdb3a2aa 2023-08-23 10:55:01,325 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,326 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,327 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,328 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/25ba0c6a926dacd93f1b3db70486e8fa/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,330 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5c07b43e4df14c1688aac8e2e1296745 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf1/5c07b43e4df14c1688aac8e2e1296745 2023-08-23 10:55:01,332 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/77f732a81ccf4e9788c073c32f36c0de to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/cf2/77f732a81ccf4e9788c073c32f36c0de 2023-08-23 10:55:01,333 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/c7bca92157974ef9ad86ff6fc5d47297 2023-08-23 10:55:01,334 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/cf2/e8383895f66145189e9852ca0c3de1d7 2023-08-23 10:55:01,334 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/17912984893b48a392eea32fff350331.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,335 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa/recovered.edits/90.seqid 2023-08-23 10:55:01,336 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/5d91a7eee88048908d1af192d24c1ebb.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,336 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/25ba0c6a926dacd93f1b3db70486e8fa 2023-08-23 10:55:01,337 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1/recovered.edits/85.seqid 2023-08-23 10:55:01,337 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/72bc35a80fd345e4b3e6b9f721a692a2.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,337 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,338 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/a9f142bcc2064dc19f5472f21e945e66.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,339 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/af3523292fe844e591c93bb7bdb3a2aa.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,340 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/c7bca92157974ef9ad86ff6fc5d47297.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,341 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/f904e288193db0dbb00e8e9fc48e0c7f/cf2/e8383895f66145189e9852ca0c3de1d7.c442a65f48f95816e9ee8b3dc3e4efb1 2023-08-23 10:55:01,343 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f/recovered.edits/90.seqid 2023-08-23 10:55:01,344 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testRecoveryAndDoubleExecution/f904e288193db0dbb00e8e9fc48e0c7f 2023-08-23 10:55:01,344 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-08-23 10:55:01,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,347 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-08-23 10:55:01,349 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-08-23 10:55:01,350 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,350 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-08-23 10:55:01,350 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788101350"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:01,350 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788101350"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:01,350 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788101350"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:01,352 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:55:01,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c442a65f48f95816e9ee8b3dc3e4efb1, NAME => 'testRecoveryAndDoubleExecution,,1692788094563.c442a65f48f95816e9ee8b3dc3e4efb1.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => f904e288193db0dbb00e8e9fc48e0c7f, NAME => 'testRecoveryAndDoubleExecution,,1692788096419.f904e288193db0dbb00e8e9fc48e0c7f.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 25ba0c6a926dacd93f1b3db70486e8fa, NAME => 'testRecoveryAndDoubleExecution,41,1692788096419.25ba0c6a926dacd93f1b3db70486e8fa.', STARTKEY => '41', ENDKEY => ''}] 2023-08-23 10:55:01,352 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-08-23 10:55:01,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788101352"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:01,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-08-23 10:55:01,356 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-08-23 10:55:01,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 63 msec 2023-08-23 10:55:01,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-08-23 10:55:01,397 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-08-23 10:55:01,412 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=379 (was 370) - Thread LEAK? -, OpenFileDescriptor=826 (was 747) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 175), ProcessCount=171 (was 171), AvailableMemoryMB=10761 (was 10820) 2023-08-23 10:55:01,426 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=379, OpenFileDescriptor=826, MaxFileDescriptor=60000, SystemLoadAverage=164, ProcessCount=171, AvailableMemoryMB=10761 2023-08-23 10:55:01,427 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:55:01,429 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:55:01,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:55:01,430 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,432 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:55:01,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:55:01,434 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,435 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 empty. 2023-08-23 10:55:01,435 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-23 10:55:01,443 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-08-23 10:55:01,444 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => b6d3ca9b77d92055b1c51ba66752ffe6, NAME => 'testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing b6d3ca9b77d92055b1c51ba66752ffe6, disabling compactions & flushes 2023-08-23 10:55:01,451 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. after waiting 0 ms 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,451 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,451 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:01,453 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:55:01,454 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788101454"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788101454"}]},"ts":"1692788101454"} 2023-08-23 10:55:01,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:55:01,456 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:55:01,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788101456"}]},"ts":"1692788101456"} 2023-08-23 10:55:01,457 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-08-23 10:55:01,459 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:55:01,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:55:01,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:55:01,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:55:01,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:55:01,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, ASSIGN}] 2023-08-23 10:55:01,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, ASSIGN 2023-08-23 10:55:01,462 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:01,612 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:55:01,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=b6d3ca9b77d92055b1c51ba66752ffe6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788101614"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788101614"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788101614"}]},"ts":"1692788101614"} 2023-08-23 10:55:01,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure b6d3ca9b77d92055b1c51ba66752ffe6, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:01,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b6d3ca9b77d92055b1c51ba66752ffe6, NAME => 'testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:55:01,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:01,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:01,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,771 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,772 DEBUG [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1 2023-08-23 10:55:01,772 DEBUG [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1 2023-08-23 10:55:01,772 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6d3ca9b77d92055b1c51ba66752ffe6 columnFamilyName cf1 2023-08-23 10:55:01,773 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] regionserver.HStore(310): Store=b6d3ca9b77d92055b1c51ba66752ffe6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:01,773 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,774 DEBUG [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2 2023-08-23 10:55:01,774 DEBUG [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2 2023-08-23 10:55:01,774 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6d3ca9b77d92055b1c51ba66752ffe6 columnFamilyName cf2 2023-08-23 10:55:01,774 INFO [StoreOpener-b6d3ca9b77d92055b1c51ba66752ffe6-1] regionserver.HStore(310): Store=b6d3ca9b77d92055b1c51ba66752ffe6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:01,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:01,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:01,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:55:01,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b6d3ca9b77d92055b1c51ba66752ffe6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65744431, jitterRate=-0.02033163607120514}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:01,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:01,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6., pid=127, masterSystemTime=1692788101767 2023-08-23 10:55:01,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=b6d3ca9b77d92055b1c51ba66752ffe6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788101781"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788101781"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788101781"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788101781"}]},"ts":"1692788101781"} 2023-08-23 10:55:01,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-08-23 10:55:01,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure b6d3ca9b77d92055b1c51ba66752ffe6, server=jenkins-hbase4.apache.org,43499,1692787997532 in 168 msec 2023-08-23 10:55:01,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-08-23 10:55:01,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, ASSIGN in 324 msec 2023-08-23 10:55:01,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:55:01,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788101786"}]},"ts":"1692788101786"} 2023-08-23 10:55:01,787 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-08-23 10:55:01,790 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:55:01,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 360 msec 2023-08-23 10:55:01,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:01,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,939 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:01,939 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:01,939 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:01,939 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:01,940 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,940 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,941 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:01,941 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:01,941 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,941 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:01,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,942 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:01,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:01,943 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:01,943 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,943 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:01,944 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:55:01,944 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:01,953 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/64b0767f3a8e4b368adf510ca389d5d1 2023-08-23 10:55:01,964 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/3f234851845147438aa157e5a7221501 2023-08-23 10:55:01,968 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/64b0767f3a8e4b368adf510ca389d5d1 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1 2023-08-23 10:55:01,971 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:55:01,972 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/3f234851845147438aa157e5a7221501 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501 2023-08-23 10:55:01,975 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:55:01,976 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 32ms, sequenceid=6, compaction requested=false 2023-08-23 10:55:01,976 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:01,976 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:01,976 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:01,976 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:01,976 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:01,976 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,976 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:01,976 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:01,978 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,978 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,979 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:01,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:01,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b8ebb62[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:01,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:01,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:01,982 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,982 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:01,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:01,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:01,984 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:01,984 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:01,984 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:01,985 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:01,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:01,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:01,988 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:01,988 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:01,988 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:01,988 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,988 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:01,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,991 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:01,991 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:01,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:01,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:01,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,992 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:01,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:01,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:01,994 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:01,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:01,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,000 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,001 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,101 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:02,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:02,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,115 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,115 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,116 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:02,116 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:02,116 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,116 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,117 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,118 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:02,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,119 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:02,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,119 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,119 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:02,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,120 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,120 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,120 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,120 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,120 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,121 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,122 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,122 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,122 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,123 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,128 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/122576b6a1554dc3a5d3698488940790 2023-08-23 10:55:02,138 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/9c258a219d3a4e8f8e2ab362fb3b284f 2023-08-23 10:55:02,142 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/122576b6a1554dc3a5d3698488940790 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790 2023-08-23 10:55:02,146 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:55:02,146 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/9c258a219d3a4e8f8e2ab362fb3b284f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f 2023-08-23 10:55:02,149 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:55:02,150 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 31ms, sequenceid=19, compaction requested=false 2023-08-23 10:55:02,150 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:02,150 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,150 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:02,150 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:02,150 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,150 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,150 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,152 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,152 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,152 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,152 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:02,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46c3a834[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:02,154 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:02,154 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,155 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,155 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,155 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,156 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,159 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:02,161 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:02,162 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:02,162 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:02,162 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,162 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,163 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,164 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,164 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,165 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,166 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,166 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,166 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,169 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,169 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,170 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,270 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:02,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:02,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,283 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,283 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,283 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:02,283 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:02,284 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,284 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,285 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:02,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,287 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,289 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,290 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,290 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,290 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,291 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,291 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,294 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:02,294 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:02,294 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,294 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:02,295 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:02,303 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/82d1bae42c944e6e82a2242977e1f03a 2023-08-23 10:55:02,319 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/92cfc82ca3f445ca8f086de922c8ec16 2023-08-23 10:55:02,323 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/82d1bae42c944e6e82a2242977e1f03a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a 2023-08-23 10:55:02,326 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:55:02,327 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/92cfc82ca3f445ca8f086de922c8ec16 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16 2023-08-23 10:55:02,330 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:55:02,332 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 38ms, sequenceid=32, compaction requested=true 2023-08-23 10:55:02,332 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:02,332 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,332 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,334 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,335 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:02,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@561cc246[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:02,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:02,337 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,339 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:02,339 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:02,339 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,345 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,346 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:02,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:02,349 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:02,349 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:02,349 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:02,349 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,349 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,351 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,351 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,352 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,352 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,354 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,354 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,355 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,358 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,358 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,359 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,459 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:02,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:02,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,472 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,472 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,472 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:02,472 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:02,472 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,472 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,474 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,474 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,474 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,474 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:02,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:02,476 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:02,476 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,476 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:02,477 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:02,478 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,478 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,478 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,478 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,479 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,479 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,485 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/184f63d6b5204e3eb3051aaa79bd537c 2023-08-23 10:55:02,496 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/67fd4fe0915648088afe8846a4478f42 2023-08-23 10:55:02,499 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/184f63d6b5204e3eb3051aaa79bd537c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c 2023-08-23 10:55:02,502 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:55:02,503 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/67fd4fe0915648088afe8846a4478f42 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42 2023-08-23 10:55:02,506 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:55:02,507 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 31ms, sequenceid=45, compaction requested=true 2023-08-23 10:55:02,507 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:02,507 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,507 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:02,508 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:02,508 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,508 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,508 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,510 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,510 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,510 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,510 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,512 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:02,512 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69fbba41[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:02,512 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:02,512 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,513 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,513 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,513 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:02,514 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,514 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:02,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,514 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:02,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,516 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,516 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:02,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:02,519 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:02,519 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:02,519 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:02,519 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,519 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,522 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,524 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,525 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,526 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,526 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-08-23 10:55:02,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,531 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,532 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,533 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,632 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:02,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:02,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,646 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:02,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:02,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,646 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,647 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,647 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,648 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,647 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,648 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,648 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,650 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,650 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,650 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:02,650 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:02,651 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,653 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,653 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,653 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,653 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,653 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,658 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/32e1519b9690401486d3026b6ede0e2c 2023-08-23 10:55:02,669 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/96f145267c2a45c59ecf4cf88dc44b52 2023-08-23 10:55:02,672 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/32e1519b9690401486d3026b6ede0e2c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c 2023-08-23 10:55:02,675 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:55:02,675 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/96f145267c2a45c59ecf4cf88dc44b52 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52 2023-08-23 10:55:02,678 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:55:02,679 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 29ms, sequenceid=58, compaction requested=true 2023-08-23 10:55:02,679 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:02,679 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:02,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:02,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,679 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,681 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,681 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,681 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,681 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c059b7[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:02,683 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:02,683 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,685 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:02,686 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,686 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,686 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,686 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,688 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:02,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:02,691 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:02,691 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:02,691 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:02,691 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,691 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,693 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,694 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,694 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,696 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,696 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,696 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,699 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,698 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,699 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,700 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,700 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,700 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,700 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,700 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,799 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:02,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:02,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,812 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,812 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,812 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-08-23 10:55:02,812 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-08-23 10:55:02,812 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,812 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,814 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,814 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,814 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,814 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:02,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,815 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:02,816 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:02,816 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,816 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. started... 2023-08-23 10:55:02,816 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:02,819 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,819 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,819 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,819 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,820 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,820 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,825 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/2a41a49fde3f467ba3e4400e4041a0ae 2023-08-23 10:55:02,836 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/7f766c970f8241e398123b8635093a29 2023-08-23 10:55:02,839 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/2a41a49fde3f467ba3e4400e4041a0ae as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae 2023-08-23 10:55:02,842 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:55:02,843 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/7f766c970f8241e398123b8635093a29 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29 2023-08-23 10:55:02,846 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:55:02,846 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 30ms, sequenceid=71, compaction requested=true 2023-08-23 10:55:02,847 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:02,847 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:02,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:02,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:02,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-08-23 10:55:02,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-08-23 10:55:02,848 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,848 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,849 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,849 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:02,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-08-23 10:55:02,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c99c373[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:02,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-08-23 10:55:02,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,855 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-08-23 10:55:02,855 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:02,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,856 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,857 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,857 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-08-23 10:55:02,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:02,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-08-23 10:55:02,860 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-08-23 10:55:02,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:02,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:02,860 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,860 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:02,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:02,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,862 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,862 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,863 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,863 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,864 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:02,865 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,865 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,872 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-08-23 10:55:02,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:02,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:02,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,873 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-08-23 10:55:02,912 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:55:02,935 ERROR [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 286f5afc-c3f2-4f9a-8396-78cfba18066f) service to localhost/127.0.0.1:34277 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:55:02,973 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:02,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:02,982 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,982 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b6d3ca9b77d92055b1c51ba66752ffe6, daughterA=0db0450f4e7c05516f9d244c7905be88, daughterB=b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:02,984 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b6d3ca9b77d92055b1c51ba66752ffe6, daughterA=0db0450f4e7c05516f9d244c7905be88, daughterB=b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:02,984 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b6d3ca9b77d92055b1c51ba66752ffe6, daughterA=0db0450f4e7c05516f9d244c7905be88, daughterB=b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:02,984 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b6d3ca9b77d92055b1c51ba66752ffe6, daughterA=0db0450f4e7c05516f9d244c7905be88, daughterB=b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:02,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, UNASSIGN}] 2023-08-23 10:55:02,986 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, UNASSIGN 2023-08-23 10:55:02,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=b6d3ca9b77d92055b1c51ba66752ffe6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:02,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788102986"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788102986"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788102986"}]},"ts":"1692788102986"} 2023-08-23 10:55:02,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure b6d3ca9b77d92055b1c51ba66752ffe6, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:03,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b6d3ca9b77d92055b1c51ba66752ffe6, disabling compactions & flushes 2023-08-23 10:55:03,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:03,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:03,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. after waiting 0 ms 2023-08-23 10:55:03,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:03,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b6d3ca9b77d92055b1c51ba66752ffe6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:55:03,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/da10d007cae94cf2bed9e70ee2a4cf14 2023-08-23 10:55:03,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/aa549af5378b4f1f91c2d2eb55f4582a 2023-08-23 10:55:03,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf1/da10d007cae94cf2bed9e70ee2a4cf14 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14 2023-08-23 10:55:03,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:55:03,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.tmp/cf2/aa549af5378b4f1f91c2d2eb55f4582a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a 2023-08-23 10:55:03,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:55:03,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b6d3ca9b77d92055b1c51ba66752ffe6 in 33ms, sequenceid=82, compaction requested=true 2023-08-23 10:55:03,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:55:03,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:03,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. 2023-08-23 10:55:03,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b6d3ca9b77d92055b1c51ba66752ffe6: 2023-08-23 10:55:03,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=b6d3ca9b77d92055b1c51ba66752ffe6, regionState=CLOSED 2023-08-23 10:55:03,180 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788103179"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788103179"}]},"ts":"1692788103179"} 2023-08-23 10:55:03,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-08-23 10:55:03,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure b6d3ca9b77d92055b1c51ba66752ffe6, server=jenkins-hbase4.apache.org,43499,1692787997532 in 194 msec 2023-08-23 10:55:03,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-08-23 10:55:03,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b6d3ca9b77d92055b1c51ba66752ffe6, UNASSIGN in 197 msec 2023-08-23 10:55:03,193 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=b6d3ca9b77d92055b1c51ba66752ffe6, threads=14 2023-08-23 10:55:03,193 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,193 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,193 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,193 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,193 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,194 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,197 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,199 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,199 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,199 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,201 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,201 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,201 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,201 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,204 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16, top=false 2023-08-23 10:55:03,205 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52, top=false 2023-08-23 10:55:03,206 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790, top=false 2023-08-23 10:55:03,211 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29, top=false 2023-08-23 10:55:03,211 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501, top=false 2023-08-23 10:55:03,213 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c, top=false 2023-08-23 10:55:03,214 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f, top=false 2023-08-23 10:55:03,216 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a, top=false 2023-08-23 10:55:03,217 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae, top=false 2023-08-23 10:55:03,218 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a, top=false 2023-08-23 10:55:03,219 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1, top=false 2023-08-23 10:55:03,222 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c, top=false 2023-08-23 10:55:03,223 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14, top=false 2023-08-23 10:55:03,223 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42, top=false 2023-08-23 10:55:03,242 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,245 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,250 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,252 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,253 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,253 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,255 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,256 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,261 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,262 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,265 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,270 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,273 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,275 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/.splits/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14 for child: 0db0450f4e7c05516f9d244c7905be88, parent: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,277 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,277 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,277 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,278 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,279 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,279 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,279 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,279 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,281 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14 for region: b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:03,281 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region b6d3ca9b77d92055b1c51ba66752ffe6 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-08-23 10:55:03,316 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:55:03,318 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:55:03,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1692788103320"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1692788103320"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1692788103320"}]},"ts":"1692788103320"} 2023-08-23 10:55:03,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788103320"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788103320"}]},"ts":"1692788103320"} 2023-08-23 10:55:03,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788103320"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788103320"}]},"ts":"1692788103320"} 2023-08-23 10:55:03,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, ASSIGN}] 2023-08-23 10:55:03,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, ASSIGN 2023-08-23 10:55:03,324 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:03,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, ASSIGN 2023-08-23 10:55:03,325 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:03,474 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:55:03,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b5c15dfcae2b395bc3022bd4bad59221, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103476"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788103476"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788103476"}]},"ts":"1692788103476"} 2023-08-23 10:55:03,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure b5c15dfcae2b395bc3022bd4bad59221, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:03,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=0db0450f4e7c05516f9d244c7905be88, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788103480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788103480"}]},"ts":"1692788103480"} 2023-08-23 10:55:03,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 0db0450f4e7c05516f9d244c7905be88, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:03,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:03,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b5c15dfcae2b395bc3022bd4bad59221, NAME => 'testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.', STARTKEY => '71', ENDKEY => ''} 2023-08-23 10:55:03,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:03,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:03,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,633 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,635 DEBUG [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf1 2023-08-23 10:55:03,635 DEBUG [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf1 2023-08-23 10:55:03,635 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5c15dfcae2b395bc3022bd4bad59221 columnFamilyName cf1 2023-08-23 10:55:03,636 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] regionserver.HStore(310): Store=b5c15dfcae2b395bc3022bd4bad59221/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:03,636 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,637 DEBUG [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf2 2023-08-23 10:55:03,637 DEBUG [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf2 2023-08-23 10:55:03,637 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5c15dfcae2b395bc3022bd4bad59221 columnFamilyName cf2 2023-08-23 10:55:03,638 INFO [StoreOpener-b5c15dfcae2b395bc3022bd4bad59221-1] regionserver.HStore(310): Store=b5c15dfcae2b395bc3022bd4bad59221/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:03,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:03,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:03,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b5c15dfcae2b395bc3022bd4bad59221; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75234959, jitterRate=0.12108825147151947}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:03,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b5c15dfcae2b395bc3022bd4bad59221: 2023-08-23 10:55:03,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221., pid=133, masterSystemTime=1692788103629 2023-08-23 10:55:03,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:03,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:03,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:03,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0db0450f4e7c05516f9d244c7905be88, NAME => 'testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.', STARTKEY => '', ENDKEY => '71'} 2023-08-23 10:55:03,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b5c15dfcae2b395bc3022bd4bad59221, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:03,645 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103644"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788103644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788103644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788103644"}]},"ts":"1692788103644"} 2023-08-23 10:55:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,647 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,648 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1 2023-08-23 10:55:03,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-08-23 10:55:03,648 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1 2023-08-23 10:55:03,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure b5c15dfcae2b395bc3022bd4bad59221, server=jenkins-hbase4.apache.org,43499,1692787997532 in 169 msec 2023-08-23 10:55:03,648 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0db0450f4e7c05516f9d244c7905be88 columnFamilyName cf1 2023-08-23 10:55:03,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, ASSIGN in 325 msec 2023-08-23 10:55:03,655 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790 2023-08-23 10:55:03,659 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c 2023-08-23 10:55:03,662 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae 2023-08-23 10:55:03,665 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c 2023-08-23 10:55:03,668 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1 2023-08-23 10:55:03,671 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a 2023-08-23 10:55:03,674 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14 2023-08-23 10:55:03,674 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(310): Store=0db0450f4e7c05516f9d244c7905be88/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:03,674 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,675 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2 2023-08-23 10:55:03,675 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2 2023-08-23 10:55:03,676 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0db0450f4e7c05516f9d244c7905be88 columnFamilyName cf2 2023-08-23 10:55:03,681 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501 2023-08-23 10:55:03,683 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42 2023-08-23 10:55:03,686 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29 2023-08-23 10:55:03,689 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16 2023-08-23 10:55:03,691 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52 2023-08-23 10:55:03,694 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f 2023-08-23 10:55:03,697 DEBUG [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a 2023-08-23 10:55:03,697 INFO [StoreOpener-0db0450f4e7c05516f9d244c7905be88-1] regionserver.HStore(310): Store=0db0450f4e7c05516f9d244c7905be88/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:03,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:03,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:03,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0db0450f4e7c05516f9d244c7905be88; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61858297, jitterRate=-0.0782395452260971}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:03,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0db0450f4e7c05516f9d244c7905be88: 2023-08-23 10:55:03,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88., pid=134, masterSystemTime=1692788103629 2023-08-23 10:55:03,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:03,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): 0db0450f4e7c05516f9d244c7905be88/cf1 is initiating minor compaction (all files) 2023-08-23 10:55:03,704 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): 0db0450f4e7c05516f9d244c7905be88/cf2 is initiating minor compaction (all files) 2023-08-23 10:55:03,704 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0db0450f4e7c05516f9d244c7905be88/cf1 in testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:03,704 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0db0450f4e7c05516f9d244c7905be88/cf2 in testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:03,705 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/.tmp, totalSize=34.9 K 2023-08-23 10:55:03,705 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/.tmp, totalSize=34.9 K 2023-08-23 10:55:03,705 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788101933 2023-08-23 10:55:03,705 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788101933 2023-08-23 10:55:03,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:03,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:03,706 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788102103 2023-08-23 10:55:03,706 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788102103 2023-08-23 10:55:03,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=0db0450f4e7c05516f9d244c7905be88, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103706"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788103706"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788103706"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788103706"}]},"ts":"1692788103706"} 2023-08-23 10:55:03,706 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788102271 2023-08-23 10:55:03,706 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788102271 2023-08-23 10:55:03,707 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788102461 2023-08-23 10:55:03,707 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788102461 2023-08-23 10:55:03,707 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788102634 2023-08-23 10:55:03,707 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788102634 2023-08-23 10:55:03,707 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788102801 2023-08-23 10:55:03,708 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788102801 2023-08-23 10:55:03,708 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788102975 2023-08-23 10:55:03,708 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788102975 2023-08-23 10:55:03,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-08-23 10:55:03,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 0db0450f4e7c05516f9d244c7905be88, server=jenkins-hbase4.apache.org,43499,1692787997532 in 227 msec 2023-08-23 10:55:03,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-08-23 10:55:03,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, ASSIGN in 387 msec 2023-08-23 10:55:03,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=b6d3ca9b77d92055b1c51ba66752ffe6, daughterA=0db0450f4e7c05516f9d244c7905be88, daughterB=b5c15dfcae2b395bc3022bd4bad59221 in 730 msec 2023-08-23 10:55:03,732 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0db0450f4e7c05516f9d244c7905be88#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:03,733 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): 0db0450f4e7c05516f9d244c7905be88#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:03,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43499] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:38534 deadline: 1692788163737, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6. is not online on jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,752 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/.tmp/cf1/1fc1555318dc4f8391f15f9e4207dc0c as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/1fc1555318dc4f8391f15f9e4207dc0c 2023-08-23 10:55:03,754 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/.tmp/cf2/3b98ac8461fe474e940feb1af5e877b6 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/3b98ac8461fe474e940feb1af5e877b6 2023-08-23 10:55:03,756 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0db0450f4e7c05516f9d244c7905be88/cf1 of 0db0450f4e7c05516f9d244c7905be88 into 1fc1555318dc4f8391f15f9e4207dc0c(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:03,756 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0db0450f4e7c05516f9d244c7905be88: 2023-08-23 10:55:03,757 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88., storeName=0db0450f4e7c05516f9d244c7905be88/cf1, priority=9, startTime=1692788103703; duration=0sec 2023-08-23 10:55:03,757 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:03,758 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0db0450f4e7c05516f9d244c7905be88/cf2 of 0db0450f4e7c05516f9d244c7905be88 into 3b98ac8461fe474e940feb1af5e877b6(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:03,758 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0db0450f4e7c05516f9d244c7905be88: 2023-08-23 10:55:03,758 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88., storeName=0db0450f4e7c05516f9d244c7905be88/cf2, priority=9, startTime=1692788103704; duration=0sec 2023-08-23 10:55:03,758 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:03,850 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-08-23 10:55:03,853 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:55:03,854 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-08-23 10:55:03,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-08-23 10:55:03,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:03,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-23 10:55:03,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788103857"}]},"ts":"1692788103857"} 2023-08-23 10:55:03,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-08-23 10:55:03,860 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-08-23 10:55:03,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, UNASSIGN}] 2023-08-23 10:55:03,861 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, UNASSIGN 2023-08-23 10:55:03,861 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b5c15dfcae2b395bc3022bd4bad59221, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103861"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788103861"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788103861"}]},"ts":"1692788103861"} 2023-08-23 10:55:03,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure b5c15dfcae2b395bc3022bd4bad59221, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:03,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, UNASSIGN 2023-08-23 10:55:03,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=0db0450f4e7c05516f9d244c7905be88, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:03,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788103864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788103864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788103864"}]},"ts":"1692788103864"} 2023-08-23 10:55:03,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 0db0450f4e7c05516f9d244c7905be88, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:03,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-23 10:55:04,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:04,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0db0450f4e7c05516f9d244c7905be88, disabling compactions & flushes 2023-08-23 10:55:04,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:04,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:04,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. after waiting 0 ms 2023-08-23 10:55:04,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:04,015 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14] to archive 2023-08-23 10:55:04,015 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:04,017 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-64b0767f3a8e4b368adf510ca389d5d1 2023-08-23 10:55:04,018 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-122576b6a1554dc3a5d3698488940790 2023-08-23 10:55:04,019 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-82d1bae42c944e6e82a2242977e1f03a 2023-08-23 10:55:04,020 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-184f63d6b5204e3eb3051aaa79bd537c 2023-08-23 10:55:04,021 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-32e1519b9690401486d3026b6ede0e2c 2023-08-23 10:55:04,022 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-2a41a49fde3f467ba3e4400e4041a0ae 2023-08-23 10:55:04,023 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-da10d007cae94cf2bed9e70ee2a4cf14 2023-08-23 10:55:04,025 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a] to archive 2023-08-23 10:55:04,025 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:04,027 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-3f234851845147438aa157e5a7221501 2023-08-23 10:55:04,028 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-9c258a219d3a4e8f8e2ab362fb3b284f 2023-08-23 10:55:04,029 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-92cfc82ca3f445ca8f086de922c8ec16 2023-08-23 10:55:04,030 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-67fd4fe0915648088afe8846a4478f42 2023-08-23 10:55:04,030 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-96f145267c2a45c59ecf4cf88dc44b52 2023-08-23 10:55:04,031 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-7f766c970f8241e398123b8635093a29 2023-08-23 10:55:04,032 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/testSplitTableRegionEmptyDaughter=b6d3ca9b77d92055b1c51ba66752ffe6-aa549af5378b4f1f91c2d2eb55f4582a 2023-08-23 10:55:04,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:55:04,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:04,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88. 2023-08-23 10:55:04,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0db0450f4e7c05516f9d244c7905be88: 2023-08-23 10:55:04,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:04,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:04,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b5c15dfcae2b395bc3022bd4bad59221, disabling compactions & flushes 2023-08-23 10:55:04,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:04,039 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=0db0450f4e7c05516f9d244c7905be88, regionState=CLOSED 2023-08-23 10:55:04,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:04,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. after waiting 0 ms 2023-08-23 10:55:04,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:04,039 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788104039"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788104039"}]},"ts":"1692788104039"} 2023-08-23 10:55:04,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-08-23 10:55:04,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 0db0450f4e7c05516f9d244c7905be88, server=jenkins-hbase4.apache.org,43499,1692787997532 in 174 msec 2023-08-23 10:55:04,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-08-23 10:55:04,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:04,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221. 2023-08-23 10:55:04,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b5c15dfcae2b395bc3022bd4bad59221: 2023-08-23 10:55:04,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0db0450f4e7c05516f9d244c7905be88, UNASSIGN in 182 msec 2023-08-23 10:55:04,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:04,044 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b5c15dfcae2b395bc3022bd4bad59221, regionState=CLOSED 2023-08-23 10:55:04,044 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1692788104044"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788104044"}]},"ts":"1692788104044"} 2023-08-23 10:55:04,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-08-23 10:55:04,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure b5c15dfcae2b395bc3022bd4bad59221, server=jenkins-hbase4.apache.org,43499,1692787997532 in 184 msec 2023-08-23 10:55:04,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-08-23 10:55:04,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5c15dfcae2b395bc3022bd4bad59221, UNASSIGN in 188 msec 2023-08-23 10:55:04,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788104049"}]},"ts":"1692788104049"} 2023-08-23 10:55:04,050 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-08-23 10:55:04,052 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-08-23 10:55:04,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 197 msec 2023-08-23 10:55:04,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-08-23 10:55:04,159 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-08-23 10:55:04,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,162 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-23 10:55:04,163 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,167 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:04,167 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:04,167 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:04,169 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/recovered.edits] 2023-08-23 10:55:04,169 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/recovered.edits] 2023-08-23 10:55:04,169 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/recovered.edits] 2023-08-23 10:55:04,173 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/122576b6a1554dc3a5d3698488940790 2023-08-23 10:55:04,173 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/1fc1555318dc4f8391f15f9e4207dc0c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf1/1fc1555318dc4f8391f15f9e4207dc0c 2023-08-23 10:55:04,174 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/184f63d6b5204e3eb3051aaa79bd537c 2023-08-23 10:55:04,176 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/recovered.edits/88.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221/recovered.edits/88.seqid 2023-08-23 10:55:04,176 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/3b98ac8461fe474e940feb1af5e877b6 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/cf2/3b98ac8461fe474e940feb1af5e877b6 2023-08-23 10:55:04,176 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/2a41a49fde3f467ba3e4400e4041a0ae 2023-08-23 10:55:04,176 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5c15dfcae2b395bc3022bd4bad59221 2023-08-23 10:55:04,178 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/32e1519b9690401486d3026b6ede0e2c 2023-08-23 10:55:04,179 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/64b0767f3a8e4b368adf510ca389d5d1 2023-08-23 10:55:04,179 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88/recovered.edits/90.seqid 2023-08-23 10:55:04,180 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/0db0450f4e7c05516f9d244c7905be88 2023-08-23 10:55:04,181 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/82d1bae42c944e6e82a2242977e1f03a 2023-08-23 10:55:04,182 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf1/da10d007cae94cf2bed9e70ee2a4cf14 2023-08-23 10:55:04,185 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/3f234851845147438aa157e5a7221501 2023-08-23 10:55:04,187 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/67fd4fe0915648088afe8846a4478f42 2023-08-23 10:55:04,188 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/7f766c970f8241e398123b8635093a29 2023-08-23 10:55:04,190 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/92cfc82ca3f445ca8f086de922c8ec16 2023-08-23 10:55:04,191 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/96f145267c2a45c59ecf4cf88dc44b52 2023-08-23 10:55:04,193 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/9c258a219d3a4e8f8e2ab362fb3b284f 2023-08-23 10:55:04,194 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/cf2/aa549af5378b4f1f91c2d2eb55f4582a 2023-08-23 10:55:04,197 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6/recovered.edits/85.seqid 2023-08-23 10:55:04,198 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegionEmptyDaughter/b6d3ca9b77d92055b1c51ba66752ffe6 2023-08-23 10:55:04,198 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-08-23 10:55:04,200 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,202 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-08-23 10:55:04,204 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-08-23 10:55:04,205 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,205 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-08-23 10:55:04,205 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788104205"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:04,205 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788104205"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:04,205 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788104205"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:04,207 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:55:04,207 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b6d3ca9b77d92055b1c51ba66752ffe6, NAME => 'testSplitTableRegionEmptyDaughter,,1692788101430.b6d3ca9b77d92055b1c51ba66752ffe6.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0db0450f4e7c05516f9d244c7905be88, NAME => 'testSplitTableRegionEmptyDaughter,,1692788102982.0db0450f4e7c05516f9d244c7905be88.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => b5c15dfcae2b395bc3022bd4bad59221, NAME => 'testSplitTableRegionEmptyDaughter,71,1692788102982.b5c15dfcae2b395bc3022bd4bad59221.', STARTKEY => '71', ENDKEY => ''}] 2023-08-23 10:55:04,207 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-08-23 10:55:04,207 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788104207"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:04,208 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-08-23 10:55:04,210 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-08-23 10:55:04,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 50 msec 2023-08-23 10:55:04,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-08-23 10:55:04,264 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-08-23 10:55:04,278 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=379 (was 379), OpenFileDescriptor=826 (was 826), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 164), ProcessCount=171 (was 171), AvailableMemoryMB=10758 (was 10761) 2023-08-23 10:55:04,292 INFO [Listener at localhost/34585] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379, OpenFileDescriptor=826, MaxFileDescriptor=60000, SystemLoadAverage=164, ProcessCount=171, AvailableMemoryMB=10757 2023-08-23 10:55:04,292 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:55:04,296 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-08-23 10:55:04,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-08-23 10:55:04,297 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-08-23 10:55:04,299 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-08-23 10:55:04,300 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-08-23 10:55:04,301 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,302 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 empty. 2023-08-23 10:55:04,302 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-23 10:55:04,316 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-08-23 10:55:04,317 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 65454596ba29b4cf9a3215fe6dd16219, NAME => 'testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 65454596ba29b4cf9a3215fe6dd16219, disabling compactions & flushes 2023-08-23 10:55:04,327 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. after waiting 0 ms 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,327 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:04,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-08-23 10:55:04,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788104330"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788104330"}]},"ts":"1692788104330"} 2023-08-23 10:55:04,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-08-23 10:55:04,332 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-08-23 10:55:04,332 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788104332"}]},"ts":"1692788104332"} 2023-08-23 10:55:04,333 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-08-23 10:55:04,335 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-08-23 10:55:04,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-08-23 10:55:04,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-08-23 10:55:04,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-08-23 10:55:04,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-08-23 10:55:04,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, ASSIGN}] 2023-08-23 10:55:04,337 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, ASSIGN 2023-08-23 10:55:04,338 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:04,488 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-08-23 10:55:04,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=65454596ba29b4cf9a3215fe6dd16219, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788104489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788104489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788104489"}]},"ts":"1692788104489"} 2023-08-23 10:55:04,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 65454596ba29b4cf9a3215fe6dd16219, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:04,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 65454596ba29b4cf9a3215fe6dd16219, NAME => 'testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.', STARTKEY => '', ENDKEY => ''} 2023-08-23 10:55:04,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,647 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,648 DEBUG [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1 2023-08-23 10:55:04,649 DEBUG [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1 2023-08-23 10:55:04,649 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65454596ba29b4cf9a3215fe6dd16219 columnFamilyName cf1 2023-08-23 10:55:04,649 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] regionserver.HStore(310): Store=65454596ba29b4cf9a3215fe6dd16219/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:04,649 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,650 DEBUG [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2 2023-08-23 10:55:04,650 DEBUG [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2 2023-08-23 10:55:04,651 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65454596ba29b4cf9a3215fe6dd16219 columnFamilyName cf2 2023-08-23 10:55:04,651 INFO [StoreOpener-65454596ba29b4cf9a3215fe6dd16219-1] regionserver.HStore(310): Store=65454596ba29b4cf9a3215fe6dd16219/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:04,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:04,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:04,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-08-23 10:55:04,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 65454596ba29b4cf9a3215fe6dd16219; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74184614, jitterRate=0.10543689131736755}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:04,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:04,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219., pid=143, masterSystemTime=1692788104642 2023-08-23 10:55:04,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=65454596ba29b4cf9a3215fe6dd16219, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788104659"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788104659"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788104659"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788104659"}]},"ts":"1692788104659"} 2023-08-23 10:55:04,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-08-23 10:55:04,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 65454596ba29b4cf9a3215fe6dd16219, server=jenkins-hbase4.apache.org,43499,1692787997532 in 169 msec 2023-08-23 10:55:04,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-08-23 10:55:04,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, ASSIGN in 325 msec 2023-08-23 10:55:04,663 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-08-23 10:55:04,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788104663"}]},"ts":"1692788104663"} 2023-08-23 10:55:04,664 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-08-23 10:55:04,666 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-08-23 10:55:04,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 369 msec 2023-08-23 10:55:04,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:04,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:04,806 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:04,806 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,806 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:04,806 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:04,807 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,807 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,808 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,808 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,808 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,808 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,809 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,810 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,810 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,811 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:04,811 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:04,810 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,811 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:04,811 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:04,812 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=120 B heapSize=928 B 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:04,821 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/763a3ff7577f4a3f892b10e395c38b6d 2023-08-23 10:55:04,832 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/d6af94541bde42de819835fce2894d3f 2023-08-23 10:55:04,836 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/763a3ff7577f4a3f892b10e395c38b6d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d 2023-08-23 10:55:04,839 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:55:04,840 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/d6af94541bde42de819835fce2894d3f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f 2023-08-23 10:55:04,843 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:55:04,844 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 32ms, sequenceid=6, compaction requested=false 2023-08-23 10:55:04,844 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-08-23 10:55:04,844 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:04,844 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,844 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:04,844 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:04,845 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,845 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:04,845 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:04,846 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,846 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:04,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:04,846 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,847 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:04,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:04,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:04,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:04,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:04,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ca73454[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:04,848 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:04,848 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,849 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:04,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,850 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:04,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:04,850 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:04,852 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,852 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:04,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:04,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:04,852 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:04,852 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:04,853 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:04,853 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:04,853 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:04,856 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:04,856 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:04,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:04,856 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:04,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:04,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:04,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:04,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:04,859 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:04,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:04,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:04,859 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:04,859 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,861 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:04,861 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,862 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,862 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,864 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:04,864 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:04,864 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,867 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:04,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:04,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:04,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:04,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,868 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:04,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,869 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,869 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:04,869 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,869 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,869 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:04,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:04,969 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:04,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:04,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:04,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:04,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:04,981 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:04,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:04,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:04,981 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,982 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,983 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:04,983 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:04,983 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:04,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:04,984 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,984 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:04,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:04,985 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:04,985 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:04,985 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:04,985 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:04,986 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,987 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,987 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:04,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:04,987 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:04,999 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/fe5b51dcebd549c6a3d5fdfb080b395a 2023-08-23 10:55:05,010 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/4aba04db31cc41d097b9e91a14029bcf 2023-08-23 10:55:05,014 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/fe5b51dcebd549c6a3d5fdfb080b395a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a 2023-08-23 10:55:05,018 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:55:05,019 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/4aba04db31cc41d097b9e91a14029bcf as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf 2023-08-23 10:55:05,023 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf, entries=10, sequenceid=19, filesize=5.0 K 2023-08-23 10:55:05,024 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 39ms, sequenceid=19, compaction requested=false 2023-08-23 10:55:05,024 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:05,024 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:05,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:05,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,024 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,026 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,026 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:05,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58baa233[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:05,029 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:05,029 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,030 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:05,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,030 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,032 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,032 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:05,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:05,035 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:05,035 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:05,035 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:05,035 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:05,035 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,037 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,038 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,038 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,038 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,040 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,040 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,040 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,043 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,043 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:05,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:05,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,044 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,045 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,045 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,045 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,144 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:05,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:05,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:05,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:05,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:05,157 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:05,158 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,158 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:05,158 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:05,158 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,158 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,159 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,159 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,159 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,159 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,160 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,160 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,161 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,161 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,161 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,161 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:05,162 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,162 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,162 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,162 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:05,163 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,167 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,167 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,167 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,167 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,167 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,171 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/9dc488b529ee42e38d038c2120b21351 2023-08-23 10:55:05,182 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/e0306bdeeb784ecaadb57dadc89f9d1b 2023-08-23 10:55:05,185 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/9dc488b529ee42e38d038c2120b21351 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351 2023-08-23 10:55:05,188 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:55:05,189 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/e0306bdeeb784ecaadb57dadc89f9d1b as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b 2023-08-23 10:55:05,192 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b, entries=10, sequenceid=32, filesize=5.0 K 2023-08-23 10:55:05,192 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 30ms, sequenceid=32, compaction requested=true 2023-08-23 10:55:05,193 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:05,193 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,193 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:05,193 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:05,193 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,193 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,193 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,194 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,194 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,194 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,194 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:05,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d324572[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:05,196 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:05,196 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,197 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,197 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:05,197 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:05,197 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:05,198 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:05,199 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,199 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,199 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,200 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,200 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,200 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,201 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,201 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,201 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,201 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:05,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:05,204 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:05,204 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:05,204 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:05,204 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:05,204 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,205 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,206 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,206 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,207 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,208 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,208 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,209 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,211 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:05,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:05,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:05,211 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,212 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,312 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:05,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:05,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:05,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:05,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:05,325 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:05,325 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,325 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:05,325 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:05,326 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,326 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,328 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,328 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,328 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,328 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:05,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,330 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,330 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,330 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,331 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:05,331 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:05,331 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,331 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:05,331 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:05,332 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,332 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,332 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,332 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,333 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,333 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,343 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/f112f9746a7f489c8a0a5816659dca0a 2023-08-23 10:55:05,361 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/8e48ea0448864658b108f4d485a24cef 2023-08-23 10:55:05,364 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/f112f9746a7f489c8a0a5816659dca0a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a 2023-08-23 10:55:05,367 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:55:05,368 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/8e48ea0448864658b108f4d485a24cef as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef 2023-08-23 10:55:05,371 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef, entries=10, sequenceid=45, filesize=5.0 K 2023-08-23 10:55:05,371 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 40ms, sequenceid=45, compaction requested=true 2023-08-23 10:55:05,371 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:05,371 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,371 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:05,371 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:05,371 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,371 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,371 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,373 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,373 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:05,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f2287fc[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:05,375 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:05,375 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,379 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,379 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,378 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,381 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,381 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:05,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:05,385 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:05,385 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:05,385 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:05,385 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:05,385 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,387 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,387 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,388 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,388 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,389 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,390 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,391 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,391 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,397 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:05,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:05,397 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,398 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:05,398 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:05,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,398 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,399 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,498 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:05,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:05,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:05,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:05,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:05,511 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:05,511 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,511 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:05,511 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:05,511 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,511 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,512 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,512 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,513 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,513 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,513 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,514 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,514 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:05,515 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:05,515 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,515 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:05,515 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:05,516 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,517 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,524 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/992be77092c14c39b091a937d3694780 2023-08-23 10:55:05,535 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/a7cc8f32c3704ba8a9881bd102c4d431 2023-08-23 10:55:05,539 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/992be77092c14c39b091a937d3694780 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780 2023-08-23 10:55:05,541 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:55:05,542 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/a7cc8f32c3704ba8a9881bd102c4d431 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431 2023-08-23 10:55:05,545 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431, entries=10, sequenceid=58, filesize=5.0 K 2023-08-23 10:55:05,546 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 31ms, sequenceid=58, compaction requested=true 2023-08-23 10:55:05,546 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:05,546 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,546 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:05,546 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:05,546 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,546 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,546 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,549 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,549 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,550 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,550 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@580253d2[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:05,551 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:05,551 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,552 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,552 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,552 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:05,553 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,555 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:05,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:05,558 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:05,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:05,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:05,558 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:05,558 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,561 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,561 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,562 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,562 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,564 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,576 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,575 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:05,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,577 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,577 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,576 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:05,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,577 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:05,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,677 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:05,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:05,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-08-23 10:55:05,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-08-23 10:55:05,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-08-23 10:55:05,692 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-08-23 10:55:05,692 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,692 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-08-23 10:55:05,692 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-08-23 10:55:05,692 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,692 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,693 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,693 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,694 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,694 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41821,1692787997489' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-08-23 10:55:05,696 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-08-23 10:55:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37675,1692787997438' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-08-23 10:55:05,697 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-08-23 10:55:05,697 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,697 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. started... 2023-08-23 10:55:05,697 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-08-23 10:55:05,698 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,699 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,700 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,708 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/d0934704c52146a9b61759c5b2ea45d4 2023-08-23 10:55:05,722 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 2023-08-23 10:55:05,727 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/d0934704c52146a9b61759c5b2ea45d4 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4 2023-08-23 10:55:05,730 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:55:05,731 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 2023-08-23 10:55:05,734 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7, entries=10, sequenceid=71, filesize=5.0 K 2023-08-23 10:55:05,735 INFO [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 38ms, sequenceid=71, compaction requested=true 2023-08-23 10:55:05,735 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:05,735 DEBUG [rs(jenkins-hbase4.apache.org,43499,1692787997532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-08-23 10:55:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-08-23 10:55:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-08-23 10:55:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-08-23 10:55:05,737 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,737 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,737 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,737 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-08-23 10:55:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43499,1692787997532' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-08-23 10:55:05,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3443249[Count = 0] remaining members to acquire global barrier 2023-08-23 10:55:05,739 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-08-23 10:55:05,740 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,741 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43499,1692787997532' in zk 2023-08-23 10:55:05,741 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41821,1692787997489' in zk 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-08-23 10:55:05,741 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37675,1692787997438' in zk 2023-08-23 10:55:05,745 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,745 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,43499,1692787997532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-08-23 10:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,37675,1692787997438' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,746 DEBUG [member: 'jenkins-hbase4.apache.org,41821,1692787997489' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-08-23 10:55:05,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43499,1692787997532': 2023-08-23 10:55:05,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43499,1692787997532' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-08-23 10:55:05,749 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-08-23 10:55:05,749 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-08-23 10:55:05,749 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-08-23 10:55:05,749 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-08-23 10:55:05,749 INFO [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-08-23 10:55:05,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-08-23 10:55:05,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,751 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-08-23 10:55:05,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,752 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-08-23 10:55:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-08-23 10:55:05,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-08-23 10:55:05,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,755 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-08-23 10:55:05,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-08-23 10:55:05,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-08-23 10:55:05,759 DEBUG [(jenkins-hbase4.apache.org,44301,1692787996219)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-08-23 10:55:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,758 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:05,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,759 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-08-23 10:55:05,760 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-08-23 10:55:05,760 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,760 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-08-23 10:55:05,760 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-08-23 10:55:05,859 DEBUG [Listener at localhost/34585] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-08-23 10:55:05,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-08-23 10:55:05,868 DEBUG [Listener at localhost/34585] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,868 DEBUG [Listener at localhost/34585] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=65454596ba29b4cf9a3215fe6dd16219, daughterA=1682b5e763ee61ea2024b41542bfb8c3, daughterB=3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:05,869 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=65454596ba29b4cf9a3215fe6dd16219, daughterA=1682b5e763ee61ea2024b41542bfb8c3, daughterB=3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:05,870 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=65454596ba29b4cf9a3215fe6dd16219, daughterA=1682b5e763ee61ea2024b41542bfb8c3, daughterB=3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:05,870 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=65454596ba29b4cf9a3215fe6dd16219, daughterA=1682b5e763ee61ea2024b41542bfb8c3, daughterB=3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:05,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, UNASSIGN}] 2023-08-23 10:55:05,872 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, UNASSIGN 2023-08-23 10:55:05,872 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=65454596ba29b4cf9a3215fe6dd16219, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:05,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788105872"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788105872"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788105872"}]},"ts":"1692788105872"} 2023-08-23 10:55:05,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 65454596ba29b4cf9a3215fe6dd16219, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:06,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 65454596ba29b4cf9a3215fe6dd16219, disabling compactions & flushes 2023-08-23 10:55:06,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:06,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:06,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. after waiting 0 ms 2023-08-23 10:55:06,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:06,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 65454596ba29b4cf9a3215fe6dd16219 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-08-23 10:55:06,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/714ad1c3ed414dfd9a3c5e1015f26034 2023-08-23 10:55:06,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/7cb28e218aed4c3db8c27d3230e2da33 2023-08-23 10:55:06,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf1/714ad1c3ed414dfd9a3c5e1015f26034 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034 2023-08-23 10:55:06,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:55:06,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/.tmp/cf2/7cb28e218aed4c3db8c27d3230e2da33 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33 2023-08-23 10:55:06,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33, entries=8, sequenceid=82, filesize=5.0 K 2023-08-23 10:55:06,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 65454596ba29b4cf9a3215fe6dd16219 in 35ms, sequenceid=82, compaction requested=true 2023-08-23 10:55:06,065 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-08-23 10:55:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:06,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. 2023-08-23 10:55:06,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 65454596ba29b4cf9a3215fe6dd16219: 2023-08-23 10:55:06,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,068 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=65454596ba29b4cf9a3215fe6dd16219, regionState=CLOSED 2023-08-23 10:55:06,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788106068"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788106068"}]},"ts":"1692788106068"} 2023-08-23 10:55:06,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-08-23 10:55:06,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 65454596ba29b4cf9a3215fe6dd16219, server=jenkins-hbase4.apache.org,43499,1692787997532 in 197 msec 2023-08-23 10:55:06,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-08-23 10:55:06,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=65454596ba29b4cf9a3215fe6dd16219, UNASSIGN in 200 msec 2023-08-23 10:55:06,081 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=65454596ba29b4cf9a3215fe6dd16219, threads=14 2023-08-23 10:55:06,081 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,081 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,081 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,081 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,081 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,082 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,083 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,083 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,438 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,442 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,453 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,456 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,459 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,459 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,489 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,493 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,494 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,501 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,514 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,517 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,517 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,523 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431 for region: 65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:06,523 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 65454596ba29b4cf9a3215fe6dd16219 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-08-23 10:55:06,574 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:55:06,575 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-08-23 10:55:06,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1692788106577"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1692788106577"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1692788106577"}]},"ts":"1692788106577"} 2023-08-23 10:55:06,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788106577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788106577"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788106577"}]},"ts":"1692788106577"} 2023-08-23 10:55:06,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788106577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788106577"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788106577"}]},"ts":"1692788106577"} 2023-08-23 10:55:06,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, ASSIGN}] 2023-08-23 10:55:06,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, ASSIGN 2023-08-23 10:55:06,581 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:06,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, ASSIGN 2023-08-23 10:55:06,583 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43499,1692787997532; forceNewPlan=false, retain=false 2023-08-23 10:55:06,732 INFO [jenkins-hbase4:44301] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-08-23 10:55:06,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=3c8a4bfa056fb1c2479880ddb752c808, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:06,734 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788106734"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788106734"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788106734"}]},"ts":"1692788106734"} 2023-08-23 10:55:06,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 3c8a4bfa056fb1c2479880ddb752c808, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:06,738 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=1682b5e763ee61ea2024b41542bfb8c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:06,738 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788106738"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788106738"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788106738"}]},"ts":"1692788106738"} 2023-08-23 10:55:06,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 1682b5e763ee61ea2024b41542bfb8c3, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:06,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3c8a4bfa056fb1c2479880ddb752c808, NAME => 'testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.', STARTKEY => '41', ENDKEY => ''} 2023-08-23 10:55:06,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:06,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:06,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,892 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,892 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1 2023-08-23 10:55:06,892 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1 2023-08-23 10:55:06,893 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c8a4bfa056fb1c2479880ddb752c808 columnFamilyName cf1 2023-08-23 10:55:06,904 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-top 2023-08-23 10:55:06,907 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-top 2023-08-23 10:55:06,910 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-top 2023-08-23 10:55:06,913 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-top 2023-08-23 10:55:06,917 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-top 2023-08-23 10:55:06,920 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-top 2023-08-23 10:55:06,923 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-top 2023-08-23 10:55:06,923 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(310): Store=3c8a4bfa056fb1c2479880ddb752c808/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:06,923 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,924 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2 2023-08-23 10:55:06,924 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2 2023-08-23 10:55:06,925 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c8a4bfa056fb1c2479880ddb752c808 columnFamilyName cf2 2023-08-23 10:55:06,936 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-top 2023-08-23 10:55:06,939 ERROR [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 0db35eca-a409-482d-977c-53ac02140840) service to localhost/127.0.0.1:34277 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:55:06,940 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-top 2023-08-23 10:55:06,943 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-top 2023-08-23 10:55:06,946 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-top 2023-08-23 10:55:06,950 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-top 2023-08-23 10:55:06,953 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-top 2023-08-23 10:55:06,957 DEBUG [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-top 2023-08-23 10:55:06,957 INFO [StoreOpener-3c8a4bfa056fb1c2479880ddb752c808-1] regionserver.HStore(310): Store=3c8a4bfa056fb1c2479880ddb752c808/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:06,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:06,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:06,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3c8a4bfa056fb1c2479880ddb752c808; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60692102, jitterRate=-0.09561720490455627}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:06,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3c8a4bfa056fb1c2479880ddb752c808: 2023-08-23 10:55:06,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808., pid=149, masterSystemTime=1692788106887 2023-08-23 10:55:06,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:06,964 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:06,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:55:06,964 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,964 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): 3c8a4bfa056fb1c2479880ddb752c808/cf1 is initiating minor compaction (all files) 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,964 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): 3c8a4bfa056fb1c2479880ddb752c808/cf2 is initiating minor compaction (all files) 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3c8a4bfa056fb1c2479880ddb752c808/cf1 in testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 3c8a4bfa056fb1c2479880ddb752c808/cf2 in testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/.tmp, totalSize=34.9 K 2023-08-23 10:55:06,964 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-top] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/.tmp, totalSize=34.9 K 2023-08-23 10:55:06,965 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788104800 2023-08-23 10:55:06,965 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1692788104800 2023-08-23 10:55:06,965 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788104970 2023-08-23 10:55:06,966 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1692788104970 2023-08-23 10:55:06,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,966 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788105146 2023-08-23 10:55:06,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:06,966 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=3c8a4bfa056fb1c2479880ddb752c808, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:06,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:06,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788106966"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788106966"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788106966"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788106966"}]},"ts":"1692788106966"} 2023-08-23 10:55:06,966 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1692788105146 2023-08-23 10:55:06,966 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788105314 2023-08-23 10:55:06,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1682b5e763ee61ea2024b41542bfb8c3, NAME => 'testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.', STARTKEY => '', ENDKEY => '41'} 2023-08-23 10:55:06,967 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1692788105314 2023-08-23 10:55:06,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-08-23 10:55:06,967 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788105500 2023-08-23 10:55:06,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:06,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-08-23 10:55:06,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:06,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:06,968 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1692788105500 2023-08-23 10:55:06,969 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788105679 2023-08-23 10:55:06,969 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1692788105679 2023-08-23 10:55:06,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-08-23 10:55:06,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 3c8a4bfa056fb1c2479880ddb752c808, server=jenkins-hbase4.apache.org,43499,1692787997532 in 233 msec 2023-08-23 10:55:06,970 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788105861 2023-08-23 10:55:06,970 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1692788105861 2023-08-23 10:55:06,970 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:06,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, ASSIGN in 389 msec 2023-08-23 10:55:06,971 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1 2023-08-23 10:55:06,971 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1 2023-08-23 10:55:06,972 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1682b5e763ee61ea2024b41542bfb8c3 columnFamilyName cf1 2023-08-23 10:55:06,981 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3c8a4bfa056fb1c2479880ddb752c808#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:06,983 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): 3c8a4bfa056fb1c2479880ddb752c808#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:06,985 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-bottom 2023-08-23 10:55:06,988 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-bottom 2023-08-23 10:55:06,991 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-bottom 2023-08-23 10:55:06,999 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-bottom 2023-08-23 10:55:07,007 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-bottom 2023-08-23 10:55:07,009 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/.tmp/cf2/fd5a2d4da7394f17a5c4ad16dcf3a3d5 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/fd5a2d4da7394f17a5c4ad16dcf3a3d5 2023-08-23 10:55:07,010 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/.tmp/cf1/3f34b1b074df457f90e4d2bf4e004538 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/3f34b1b074df457f90e4d2bf4e004538 2023-08-23 10:55:07,011 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-bottom 2023-08-23 10:55:07,015 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-bottom 2023-08-23 10:55:07,015 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(310): Store=1682b5e763ee61ea2024b41542bfb8c3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:07,016 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,016 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3c8a4bfa056fb1c2479880ddb752c808/cf2 of 3c8a4bfa056fb1c2479880ddb752c808 into fd5a2d4da7394f17a5c4ad16dcf3a3d5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:07,016 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3c8a4bfa056fb1c2479880ddb752c808: 2023-08-23 10:55:07,016 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808., storeName=3c8a4bfa056fb1c2479880ddb752c808/cf2, priority=9, startTime=1692788106964; duration=0sec 2023-08-23 10:55:07,016 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:07,016 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2 2023-08-23 10:55:07,016 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2 2023-08-23 10:55:07,017 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1682b5e763ee61ea2024b41542bfb8c3 columnFamilyName cf2 2023-08-23 10:55:07,017 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3c8a4bfa056fb1c2479880ddb752c808/cf1 of 3c8a4bfa056fb1c2479880ddb752c808 into 3f34b1b074df457f90e4d2bf4e004538(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:07,017 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3c8a4bfa056fb1c2479880ddb752c808: 2023-08-23 10:55:07,017 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808., storeName=3c8a4bfa056fb1c2479880ddb752c808/cf1, priority=9, startTime=1692788106963; duration=0sec 2023-08-23 10:55:07,017 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:07,028 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-bottom 2023-08-23 10:55:07,031 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-bottom 2023-08-23 10:55:07,034 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-bottom 2023-08-23 10:55:07,036 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-bottom 2023-08-23 10:55:07,040 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-bottom 2023-08-23 10:55:07,043 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-bottom 2023-08-23 10:55:07,049 DEBUG [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(539): loaded hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-bottom 2023-08-23 10:55:07,050 INFO [StoreOpener-1682b5e763ee61ea2024b41542bfb8c3-1] regionserver.HStore(310): Store=1682b5e763ee61ea2024b41542bfb8c3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-08-23 10:55:07,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-08-23 10:55:07,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1682b5e763ee61ea2024b41542bfb8c3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61387035, jitterRate=-0.08526189625263214}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-08-23 10:55:07,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1682b5e763ee61ea2024b41542bfb8c3: 2023-08-23 10:55:07,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3., pid=150, masterSystemTime=1692788106887 2023-08-23 10:55:07,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:07,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-08-23 10:55:07,056 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:07,056 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,056 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1912): 1682b5e763ee61ea2024b41542bfb8c3/cf2 is initiating minor compaction (all files) 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,056 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1912): 1682b5e763ee61ea2024b41542bfb8c3/cf1 is initiating minor compaction (all files) 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1682b5e763ee61ea2024b41542bfb8c3/cf2 in testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1682b5e763ee61ea2024b41542bfb8c3/cf1 in testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/.tmp, totalSize=34.9 K 2023-08-23 10:55:07,056 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-bottom] into tmpdir=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/.tmp, totalSize=34.9 K 2023-08-23 10:55:07,057 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788104800 2023-08-23 10:55:07,057 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1692788104800 2023-08-23 10:55:07,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,057 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788104970 2023-08-23 10:55:07,057 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1692788104970 2023-08-23 10:55:07,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=1682b5e763ee61ea2024b41542bfb8c3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788107057"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1692788107057"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1692788107057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1692788107057"}]},"ts":"1692788107057"} 2023-08-23 10:55:07,058 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788105146 2023-08-23 10:55:07,058 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1692788105146 2023-08-23 10:55:07,058 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788105314 2023-08-23 10:55:07,058 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1692788105314 2023-08-23 10:55:07,059 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788105500 2023-08-23 10:55:07,059 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1692788105500 2023-08-23 10:55:07,059 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788105679 2023-08-23 10:55:07,060 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1692788105679 2023-08-23 10:55:07,060 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] compactions.Compactor(207): Compacting 7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788105861 2023-08-23 10:55:07,060 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] compactions.Compactor(207): Compacting 714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1692788105861 2023-08-23 10:55:07,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-08-23 10:55:07,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 1682b5e763ee61ea2024b41542bfb8c3, server=jenkins-hbase4.apache.org,43499,1692787997532 in 321 msec 2023-08-23 10:55:07,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-08-23 10:55:07,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, ASSIGN in 480 msec 2023-08-23 10:55:07,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=65454596ba29b4cf9a3215fe6dd16219, daughterA=1682b5e763ee61ea2024b41542bfb8c3, daughterB=3c8a4bfa056fb1c2479880ddb752c808 in 1.1940 sec 2023-08-23 10:55:07,069 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1682b5e763ee61ea2024b41542bfb8c3#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:07,070 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] throttle.PressureAwareThroughputController(145): 1682b5e763ee61ea2024b41542bfb8c3#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-08-23 10:55:07,083 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/.tmp/cf2/90368d0f447e4cd2959504ef82b26929 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/90368d0f447e4cd2959504ef82b26929 2023-08-23 10:55:07,092 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1682b5e763ee61ea2024b41542bfb8c3/cf2 of 1682b5e763ee61ea2024b41542bfb8c3 into 90368d0f447e4cd2959504ef82b26929(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:07,092 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1682b5e763ee61ea2024b41542bfb8c3: 2023-08-23 10:55:07,093 INFO [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3., storeName=1682b5e763ee61ea2024b41542bfb8c3/cf2, priority=9, startTime=1692788107056; duration=0sec 2023-08-23 10:55:07,093 DEBUG [RS:2;jenkins-hbase4:43499-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:07,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43499] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:38534 deadline: 1692788167158, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219. is not online on jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,267 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:55:07,273 INFO [Listener at localhost/34585] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-08-23 10:55:07,275 DEBUG [Listener at localhost/34585] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-08-23 10:55:07,291 WARN [Listener at localhost/34585] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-08-23 10:55:07,292 INFO [Listener at localhost/34585] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-08-23 10:55:07,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-08-23 10:55:07,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-23 10:55:07,295 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788107295"}]},"ts":"1692788107295"} 2023-08-23 10:55:07,296 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-08-23 10:55:07,299 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-08-23 10:55:07,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, UNASSIGN}] 2023-08-23 10:55:07,300 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, UNASSIGN 2023-08-23 10:55:07,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=3c8a4bfa056fb1c2479880ddb752c808, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,301 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788107300"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788107300"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788107300"}]},"ts":"1692788107300"} 2023-08-23 10:55:07,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 3c8a4bfa056fb1c2479880ddb752c808, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:07,303 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, UNASSIGN 2023-08-23 10:55:07,303 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=1682b5e763ee61ea2024b41542bfb8c3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788107303"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1692788107303"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1692788107303"}]},"ts":"1692788107303"} 2023-08-23 10:55:07,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 1682b5e763ee61ea2024b41542bfb8c3, server=jenkins-hbase4.apache.org,43499,1692787997532}] 2023-08-23 10:55:07,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-23 10:55:07,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1682b5e763ee61ea2024b41542bfb8c3, disabling compactions & flushes 2023-08-23 10:55:07,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,484 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/.tmp/cf1/cdf8861a0db84a1db431a45209ddb42f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/cdf8861a0db84a1db431a45209ddb42f 2023-08-23 10:55:07,488 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1682b5e763ee61ea2024b41542bfb8c3/cf1 of 1682b5e763ee61ea2024b41542bfb8c3 into cdf8861a0db84a1db431a45209ddb42f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-08-23 10:55:07,488 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1682b5e763ee61ea2024b41542bfb8c3: 2023-08-23 10:55:07,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,488 INFO [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3., storeName=1682b5e763ee61ea2024b41542bfb8c3/cf1, priority=9, startTime=1692788107055; duration=0sec 2023-08-23 10:55:07,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,488 DEBUG [RS:2;jenkins-hbase4:43499-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-08-23 10:55:07,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. after waiting 0 ms 2023-08-23 10:55:07,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,489 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-bottom] to archive 2023-08-23 10:55:07,489 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:07,491 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,492 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,493 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,494 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,495 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,496 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,497 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,498 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-bottom, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-bottom] to archive 2023-08-23 10:55:07,498 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:07,499 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,500 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,501 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,503 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,504 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,505 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,506 DEBUG [StoreCloser-testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:55:07,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:07,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3. 2023-08-23 10:55:07,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1682b5e763ee61ea2024b41542bfb8c3: 2023-08-23 10:55:07,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:07,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3c8a4bfa056fb1c2479880ddb752c808, disabling compactions & flushes 2023-08-23 10:55:07,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:07,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:07,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. after waiting 0 ms 2023-08-23 10:55:07,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:07,511 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=1682b5e763ee61ea2024b41542bfb8c3, regionState=CLOSED 2023-08-23 10:55:07,512 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788107511"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788107511"}]},"ts":"1692788107511"} 2023-08-23 10:55:07,512 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034-top] to archive 2023-08-23 10:55:07,513 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:07,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-08-23 10:55:07,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 1682b5e763ee61ea2024b41542bfb8c3, server=jenkins-hbase4.apache.org,43499,1692787997532 in 209 msec 2023-08-23 10:55:07,514 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/763a3ff7577f4a3f892b10e395c38b6d.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=1682b5e763ee61ea2024b41542bfb8c3, UNASSIGN in 215 msec 2023-08-23 10:55:07,516 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/fe5b51dcebd549c6a3d5fdfb080b395a.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,517 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/9dc488b529ee42e38d038c2120b21351.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,518 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/f112f9746a7f489c8a0a5816659dca0a.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,519 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/992be77092c14c39b091a937d3694780.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,520 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/d0934704c52146a9b61759c5b2ea45d4.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,522 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/714ad1c3ed414dfd9a3c5e1015f26034.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,523 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7-top, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219->hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33-top] to archive 2023-08-23 10:55:07,524 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-08-23 10:55:07,525 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/d6af94541bde42de819835fce2894d3f.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,526 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/4aba04db31cc41d097b9e91a14029bcf.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,527 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/e0306bdeeb784ecaadb57dadc89f9d1b.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,528 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/8e48ea0448864658b108f4d485a24cef.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,529 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/a7cc8f32c3704ba8a9881bd102c4d431.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,530 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/2889dc5ef6fc45a68f93c27a425fbfd7.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,531 DEBUG [StoreCloser-testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/7cb28e218aed4c3db8c27d3230e2da33.65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-08-23 10:55:07,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:07,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808. 2023-08-23 10:55:07,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3c8a4bfa056fb1c2479880ddb752c808: 2023-08-23 10:55:07,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:07,536 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=3c8a4bfa056fb1c2479880ddb752c808, regionState=CLOSED 2023-08-23 10:55:07,536 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1692788107536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1692788107536"}]},"ts":"1692788107536"} 2023-08-23 10:55:07,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-08-23 10:55:07,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 3c8a4bfa056fb1c2479880ddb752c808, server=jenkins-hbase4.apache.org,43499,1692787997532 in 236 msec 2023-08-23 10:55:07,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-08-23 10:55:07,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3c8a4bfa056fb1c2479880ddb752c808, UNASSIGN in 239 msec 2023-08-23 10:55:07,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1692788107540"}]},"ts":"1692788107540"} 2023-08-23 10:55:07,541 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-08-23 10:55:07,542 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-08-23 10:55:07,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 250 msec 2023-08-23 10:55:07,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-08-23 10:55:07,598 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-08-23 10:55:07,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-08-23 10:55:07,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,601 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-23 10:55:07,606 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,606 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,606 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:07,608 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/recovered.edits] 2023-08-23 10:55:07,608 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/recovered.edits] 2023-08-23 10:55:07,608 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2, FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/recovered.edits] 2023-08-23 10:55:07,611 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/cdf8861a0db84a1db431a45209ddb42f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf1/cdf8861a0db84a1db431a45209ddb42f 2023-08-23 10:55:07,612 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/3f34b1b074df457f90e4d2bf4e004538 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf1/3f34b1b074df457f90e4d2bf4e004538 2023-08-23 10:55:07,612 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/714ad1c3ed414dfd9a3c5e1015f26034 2023-08-23 10:55:07,614 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/763a3ff7577f4a3f892b10e395c38b6d 2023-08-23 10:55:07,614 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/90368d0f447e4cd2959504ef82b26929 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/cf2/90368d0f447e4cd2959504ef82b26929 2023-08-23 10:55:07,614 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/fd5a2d4da7394f17a5c4ad16dcf3a3d5 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/cf2/fd5a2d4da7394f17a5c4ad16dcf3a3d5 2023-08-23 10:55:07,615 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/992be77092c14c39b091a937d3694780 2023-08-23 10:55:07,617 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/9dc488b529ee42e38d038c2120b21351 2023-08-23 10:55:07,617 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3/recovered.edits/90.seqid 2023-08-23 10:55:07,618 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/recovered.edits/90.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808/recovered.edits/90.seqid 2023-08-23 10:55:07,618 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/1682b5e763ee61ea2024b41542bfb8c3 2023-08-23 10:55:07,618 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/3c8a4bfa056fb1c2479880ddb752c808 2023-08-23 10:55:07,619 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/d0934704c52146a9b61759c5b2ea45d4 2023-08-23 10:55:07,620 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/f112f9746a7f489c8a0a5816659dca0a 2023-08-23 10:55:07,621 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf1/fe5b51dcebd549c6a3d5fdfb080b395a 2023-08-23 10:55:07,623 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/2889dc5ef6fc45a68f93c27a425fbfd7 2023-08-23 10:55:07,624 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/4aba04db31cc41d097b9e91a14029bcf 2023-08-23 10:55:07,625 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/7cb28e218aed4c3db8c27d3230e2da33 2023-08-23 10:55:07,626 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/8e48ea0448864658b108f4d485a24cef 2023-08-23 10:55:07,627 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431 to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/a7cc8f32c3704ba8a9881bd102c4d431 2023-08-23 10:55:07,628 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/d6af94541bde42de819835fce2894d3f 2023-08-23 10:55:07,629 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/cf2/e0306bdeeb784ecaadb57dadc89f9d1b 2023-08-23 10:55:07,631 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/recovered.edits/85.seqid to hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/archive/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219/recovered.edits/85.seqid 2023-08-23 10:55:07,632 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/.tmp/data/default/testSplitTableRegion/65454596ba29b4cf9a3215fe6dd16219 2023-08-23 10:55:07,632 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-08-23 10:55:07,634 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,635 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-08-23 10:55:07,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-08-23 10:55:07,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-08-23 10:55:07,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788107638"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:07,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788107638"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:07,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1692788107638"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:07,639 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-08-23 10:55:07,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 65454596ba29b4cf9a3215fe6dd16219, NAME => 'testSplitTableRegion,,1692788104297.65454596ba29b4cf9a3215fe6dd16219.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1682b5e763ee61ea2024b41542bfb8c3, NAME => 'testSplitTableRegion,,1692788105868.1682b5e763ee61ea2024b41542bfb8c3.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 3c8a4bfa056fb1c2479880ddb752c808, NAME => 'testSplitTableRegion,41,1692788105868.3c8a4bfa056fb1c2479880ddb752c808.', STARTKEY => '41', ENDKEY => ''}] 2023-08-23 10:55:07,639 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-08-23 10:55:07,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1692788107639"}]},"ts":"9223372036854775807"} 2023-08-23 10:55:07,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-08-23 10:55:07,642 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-08-23 10:55:07,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 43 msec 2023-08-23 10:55:07,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44301] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-08-23 10:55:07,703 INFO [Listener at localhost/34585] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-08-23 10:55:07,718 INFO [Listener at localhost/34585] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=380 (was 379) - Thread LEAK? -, OpenFileDescriptor=836 (was 826) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=167 (was 164) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=10701 (was 10757) 2023-08-23 10:55:07,719 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-08-23 10:55:07,719 INFO [Listener at localhost/34585] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-08-23 10:55:07,719 DEBUG [Listener at localhost/34585] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53986fab to 127.0.0.1:52557 2023-08-23 10:55:07,719 DEBUG [Listener at localhost/34585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,720 DEBUG [Listener at localhost/34585] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-08-23 10:55:07,720 DEBUG [Listener at localhost/34585] util.JVMClusterUtil(257): Found active master hash=282354227, stopped=false 2023-08-23 10:55:07,720 INFO [Listener at localhost/34585] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:55:07,725 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-23 10:55:07,725 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-23 10:55:07,725 INFO [Listener at localhost/34585] procedure2.ProcedureExecutor(629): Stopping 2023-08-23 10:55:07,725 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-23 10:55:07,725 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-08-23 10:55:07,725 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:55:07,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:55:07,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:55:07,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:55:07,726 DEBUG [Listener at localhost/34585] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5638e7f8 to 127.0.0.1:52557 2023-08-23 10:55:07,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-08-23 10:55:07,726 DEBUG [Listener at localhost/34585] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37675,1692787997438' ***** 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41821,1692787997489' ***** 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,43499,1692787997532' ***** 2023-08-23 10:55:07,726 INFO [Listener at localhost/34585] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-08-23 10:55:07,726 INFO [RS:1;jenkins-hbase4:41821] regionserver.HeapMemoryManager(220): Stopping 2023-08-23 10:55:07,726 INFO [RS:2;jenkins-hbase4:43499] regionserver.HeapMemoryManager(220): Stopping 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-23 10:55:07,727 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-23 10:55:07,727 INFO [RS:0;jenkins-hbase4:37675] regionserver.HeapMemoryManager(220): Stopping 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:07,727 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-23 10:55:07,727 DEBUG [RS:1;jenkins-hbase4:41821] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13f3c187 to 127.0.0.1:52557 2023-08-23 10:55:07,727 INFO [RS:0;jenkins-hbase4:37675] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-23 10:55:07,727 DEBUG [RS:1;jenkins-hbase4:41821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,727 INFO [RS:0;jenkins-hbase4:37675] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-23 10:55:07,727 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-08-23 10:55:07,727 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(3305): Received CLOSE for 03da722d073e4770f60de66574c27a90 2023-08-23 10:55:07,727 INFO [RS:2;jenkins-hbase4:43499] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-23 10:55:07,727 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-08-23 10:55:07,727 INFO [RS:2;jenkins-hbase4:43499] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-08-23 10:55:07,728 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 03da722d073e4770f60de66574c27a90, disabling compactions & flushes 2023-08-23 10:55:07,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:55:07,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:55:07,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. after waiting 0 ms 2023-08-23 10:55:07,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:55:07,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 03da722d073e4770f60de66574c27a90 1/1 column families, dataSize=78 B heapSize=488 B 2023-08-23 10:55:07,728 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:07,728 DEBUG [RS:2;jenkins-hbase4:43499] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71035186 to 127.0.0.1:52557 2023-08-23 10:55:07,729 DEBUG [RS:0;jenkins-hbase4:37675] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x048344f7 to 127.0.0.1:52557 2023-08-23 10:55:07,729 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-23 10:55:07,729 DEBUG [RS:2;jenkins-hbase4:43499] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,729 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-08-23 10:55:07,729 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-08-23 10:55:07,729 DEBUG [RS:0;jenkins-hbase4:37675] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,729 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-08-23 10:55:07,729 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,43499,1692787997532; all regions closed. 2023-08-23 10:55:07,730 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-08-23 10:55:07,730 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-08-23 10:55:07,730 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-08-23 10:55:07,730 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-08-23 10:55:07,730 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1478): Online Regions={03da722d073e4770f60de66574c27a90=hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90.} 2023-08-23 10:55:07,731 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-08-23 10:55:07,731 DEBUG [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-08-23 10:55:07,731 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1504): Waiting on 03da722d073e4770f60de66574c27a90 2023-08-23 10:55:07,752 DEBUG [RS:2;jenkins-hbase4:43499] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs 2023-08-23 10:55:07,752 INFO [RS:2;jenkins-hbase4:43499] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43499%2C1692787997532:(num 1692787999138) 2023-08-23 10:55:07,752 DEBUG [RS:2;jenkins-hbase4:43499] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,752 INFO [RS:2;jenkins-hbase4:43499] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:07,759 INFO [RS:2;jenkins-hbase4:43499] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-08-23 10:55:07,760 INFO [RS:2;jenkins-hbase4:43499] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-23 10:55:07,760 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-23 10:55:07,760 INFO [RS:2;jenkins-hbase4:43499] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-23 10:55:07,760 INFO [RS:2;jenkins-hbase4:43499] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-23 10:55:07,761 INFO [RS:2;jenkins-hbase4:43499] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43499 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43499,1692787997532 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:07,770 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:07,770 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43499,1692787997532] 2023-08-23 10:55:07,771 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43499,1692787997532; numProcessing=1 2023-08-23 10:55:07,771 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/info/30891d6b7ea74f4497eba77ea036e51f 2023-08-23 10:55:07,773 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43499,1692787997532 already deleted, retry=false 2023-08-23 10:55:07,773 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43499,1692787997532 expired; onlineServers=2 2023-08-23 10:55:07,774 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30891d6b7ea74f4497eba77ea036e51f 2023-08-23 10:55:07,783 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/rep_barrier/21bb2d159683483db1cda52bb6a8dadb 2023-08-23 10:55:07,786 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21bb2d159683483db1cda52bb6a8dadb 2023-08-23 10:55:07,800 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/table/00a41bb86170476090d2844ded93f16a 2023-08-23 10:55:07,803 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00a41bb86170476090d2844ded93f16a 2023-08-23 10:55:07,804 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/info/30891d6b7ea74f4497eba77ea036e51f as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/info/30891d6b7ea74f4497eba77ea036e51f 2023-08-23 10:55:07,807 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30891d6b7ea74f4497eba77ea036e51f 2023-08-23 10:55:07,807 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/info/30891d6b7ea74f4497eba77ea036e51f, entries=59, sequenceid=210, filesize=11.1 K 2023-08-23 10:55:07,808 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:07,808 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/rep_barrier/21bb2d159683483db1cda52bb6a8dadb as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/rep_barrier/21bb2d159683483db1cda52bb6a8dadb 2023-08-23 10:55:07,808 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:07,808 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:07,811 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21bb2d159683483db1cda52bb6a8dadb 2023-08-23 10:55:07,812 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/rep_barrier/21bb2d159683483db1cda52bb6a8dadb, entries=24, sequenceid=210, filesize=7.5 K 2023-08-23 10:55:07,812 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/.tmp/table/00a41bb86170476090d2844ded93f16a as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/table/00a41bb86170476090d2844ded93f16a 2023-08-23 10:55:07,815 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00a41bb86170476090d2844ded93f16a 2023-08-23 10:55:07,815 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/table/00a41bb86170476090d2844ded93f16a, entries=36, sequenceid=210, filesize=8.0 K 2023-08-23 10:55:07,816 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 85ms, sequenceid=210, compaction requested=false 2023-08-23 10:55:07,825 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-08-23 10:55:07,826 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:07,826 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-08-23 10:55:07,827 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-08-23 10:55:07,827 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-08-23 10:55:07,827 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-08-23 10:55:07,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:07,872 INFO [RS:2;jenkins-hbase4:43499] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,43499,1692787997532; zookeeper connection closed. 2023-08-23 10:55:07,872 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:43499-0x10038c62a350003, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:07,872 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@235c0425] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@235c0425 2023-08-23 10:55:07,931 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41821,1692787997489; all regions closed. 2023-08-23 10:55:07,931 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1504): Waiting on 03da722d073e4770f60de66574c27a90 2023-08-23 10:55:07,937 ERROR [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 286f5afc-c3f2-4f9a-8396-78cfba18066f) service to localhost/127.0.0.1:34277 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-08-23 10:55:07,938 DEBUG [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs 2023-08-23 10:55:07,938 INFO [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41821%2C1692787997489.meta:.meta(num 1692787999364) 2023-08-23 10:55:07,943 DEBUG [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs 2023-08-23 10:55:07,943 INFO [RS:1;jenkins-hbase4:41821] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41821%2C1692787997489:(num 1692787999137) 2023-08-23 10:55:07,943 DEBUG [RS:1;jenkins-hbase4:41821] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:07,943 INFO [RS:1;jenkins-hbase4:41821] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:07,943 INFO [RS:1;jenkins-hbase4:41821] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-23 10:55:07,944 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-23 10:55:07,945 INFO [RS:1;jenkins-hbase4:41821] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41821 2023-08-23 10:55:07,947 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:07,947 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41821,1692787997489 2023-08-23 10:55:07,947 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:07,947 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41821,1692787997489] 2023-08-23 10:55:07,947 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41821,1692787997489; numProcessing=2 2023-08-23 10:55:07,952 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41821,1692787997489 already deleted, retry=false 2023-08-23 10:55:07,952 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41821,1692787997489 expired; onlineServers=1 2023-08-23 10:55:07,958 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-08-23 10:55:07,958 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-08-23 10:55:08,052 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,052 INFO [RS:1;jenkins-hbase4:41821] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41821,1692787997489; zookeeper connection closed. 2023-08-23 10:55:08,052 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:41821-0x10038c62a350002, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,052 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2595c20] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2595c20 2023-08-23 10:55:08,131 DEBUG [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1504): Waiting on 03da722d073e4770f60de66574c27a90 2023-08-23 10:55:08,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/.tmp/info/2ba4fae7b234466b9b6428a65fa15961 2023-08-23 10:55:08,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/.tmp/info/2ba4fae7b234466b9b6428a65fa15961 as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/info/2ba4fae7b234466b9b6428a65fa15961 2023-08-23 10:55:08,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/info/2ba4fae7b234466b9b6428a65fa15961, entries=2, sequenceid=6, filesize=4.8 K 2023-08-23 10:55:08,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 03da722d073e4770f60de66574c27a90 in 446ms, sequenceid=6, compaction requested=false 2023-08-23 10:55:08,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/data/hbase/namespace/03da722d073e4770f60de66574c27a90/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-08-23 10:55:08,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-08-23 10:55:08,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:55:08,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 03da722d073e4770f60de66574c27a90: 2023-08-23 10:55:08,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1692787999658.03da722d073e4770f60de66574c27a90. 2023-08-23 10:55:08,332 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37675,1692787997438; all regions closed. 2023-08-23 10:55:08,337 DEBUG [RS:0;jenkins-hbase4:37675] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/oldWALs 2023-08-23 10:55:08,337 INFO [RS:0;jenkins-hbase4:37675] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37675%2C1692787997438:(num 1692787999137) 2023-08-23 10:55:08,337 DEBUG [RS:0;jenkins-hbase4:37675] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:08,337 INFO [RS:0;jenkins-hbase4:37675] regionserver.LeaseManager(133): Closed leases 2023-08-23 10:55:08,338 INFO [RS:0;jenkins-hbase4:37675] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-08-23 10:55:08,338 INFO [RS:0;jenkins-hbase4:37675] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-08-23 10:55:08,338 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-23 10:55:08,338 INFO [RS:0;jenkins-hbase4:37675] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-08-23 10:55:08,338 INFO [RS:0;jenkins-hbase4:37675] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-08-23 10:55:08,339 INFO [RS:0;jenkins-hbase4:37675] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37675 2023-08-23 10:55:08,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-08-23 10:55:08,340 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37675,1692787997438 2023-08-23 10:55:08,341 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37675,1692787997438] 2023-08-23 10:55:08,341 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37675,1692787997438; numProcessing=3 2023-08-23 10:55:08,343 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37675,1692787997438 already deleted, retry=false 2023-08-23 10:55:08,343 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37675,1692787997438 expired; onlineServers=0 2023-08-23 10:55:08,343 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,44301,1692787996219' ***** 2023-08-23 10:55:08,343 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-08-23 10:55:08,344 DEBUG [M:0;jenkins-hbase4:44301] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51e96808, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-08-23 10:55:08,344 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,44301,1692787996219 2023-08-23 10:55:08,344 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,44301,1692787996219; all regions closed. 2023-08-23 10:55:08,344 DEBUG [M:0;jenkins-hbase4:44301] ipc.AbstractRpcClient(494): Stopping rpc client 2023-08-23 10:55:08,344 DEBUG [M:0;jenkins-hbase4:44301] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-08-23 10:55:08,344 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-08-23 10:55:08,345 DEBUG [M:0;jenkins-hbase4:44301] cleaner.HFileCleaner(317): Stopping file delete threads 2023-08-23 10:55:08,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692787998764] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1692787998764,5,FailOnTimeoutGroup] 2023-08-23 10:55:08,345 INFO [M:0;jenkins-hbase4:44301] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-08-23 10:55:08,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692787998770] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1692787998770,5,FailOnTimeoutGroup] 2023-08-23 10:55:08,345 INFO [M:0;jenkins-hbase4:44301] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-08-23 10:55:08,345 INFO [M:0;jenkins-hbase4:44301] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-08-23 10:55:08,345 DEBUG [M:0;jenkins-hbase4:44301] master.HMaster(1512): Stopping service threads 2023-08-23 10:55:08,345 INFO [M:0;jenkins-hbase4:44301] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-08-23 10:55:08,345 ERROR [M:0;jenkins-hbase4:44301] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-08-23 10:55:08,345 INFO [M:0;jenkins-hbase4:44301] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-08-23 10:55:08,345 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-08-23 10:55:08,346 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-08-23 10:55:08,346 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-08-23 10:55:08,346 DEBUG [M:0;jenkins-hbase4:44301] zookeeper.ZKUtil(398): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-08-23 10:55:08,346 WARN [M:0;jenkins-hbase4:44301] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-08-23 10:55:08,346 INFO [M:0;jenkins-hbase4:44301] assignment.AssignmentManager(315): Stopping assignment manager 2023-08-23 10:55:08,346 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-08-23 10:55:08,346 INFO [M:0;jenkins-hbase4:44301] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-08-23 10:55:08,346 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-08-23 10:55:08,346 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:55:08,346 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:55:08,346 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-08-23 10:55:08,346 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:55:08,347 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.13 KB heapSize=719.03 KB 2023-08-23 10:55:08,363 INFO [M:0;jenkins-hbase4:44301] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.13 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/91c033dfc0ea4e16a666df5c89ddc04d 2023-08-23 10:55:08,366 INFO [M:0;jenkins-hbase4:44301] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 91c033dfc0ea4e16a666df5c89ddc04d 2023-08-23 10:55:08,367 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/91c033dfc0ea4e16a666df5c89ddc04d as hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/91c033dfc0ea4e16a666df5c89ddc04d 2023-08-23 10:55:08,370 INFO [M:0;jenkins-hbase4:44301] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 91c033dfc0ea4e16a666df5c89ddc04d 2023-08-23 10:55:08,370 INFO [M:0;jenkins-hbase4:44301] regionserver.HStore(1080): Added hdfs://localhost:34277/user/jenkins/test-data/cc0fb086-0a72-bde7-a070-d29d14deab5e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/91c033dfc0ea4e16a666df5c89ddc04d, entries=156, sequenceid=1282, filesize=34.9 K 2023-08-23 10:55:08,371 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegion(2948): Finished flush of dataSize ~608.13 KB/622728, heapSize ~719.02 KB/736272, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 25ms, sequenceid=1282, compaction requested=false 2023-08-23 10:55:08,372 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-08-23 10:55:08,372 DEBUG [M:0;jenkins-hbase4:44301] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-08-23 10:55:08,377 INFO [M:0;jenkins-hbase4:44301] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-08-23 10:55:08,377 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-08-23 10:55:08,378 INFO [M:0;jenkins-hbase4:44301] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44301 2023-08-23 10:55:08,379 DEBUG [M:0;jenkins-hbase4:44301] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,44301,1692787996219 already deleted, retry=false 2023-08-23 10:55:08,441 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,441 INFO [RS:0;jenkins-hbase4:37675] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37675,1692787997438; zookeeper connection closed. 2023-08-23 10:55:08,442 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): regionserver:37675-0x10038c62a350001, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,442 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2a67273c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2a67273c 2023-08-23 10:55:08,442 INFO [Listener at localhost/34585] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-08-23 10:55:08,542 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,542 INFO [M:0;jenkins-hbase4:44301] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,44301,1692787996219; zookeeper connection closed. 2023-08-23 10:55:08,542 DEBUG [Listener at localhost/34585-EventThread] zookeeper.ZKWatcher(600): master:44301-0x10038c62a350000, quorum=127.0.0.1:52557, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-08-23 10:55:08,544 WARN [Listener at localhost/34585] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-23 10:55:08,548 INFO [Listener at localhost/34585] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-23 10:55:08,655 WARN [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-08-23 10:55:08,655 WARN [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 44620c13-9adc-474d-8c34-bedbb2aa5c0d) service to localhost/127.0.0.1:34277 2023-08-23 10:55:08,661 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data5/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,661 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data6/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,663 WARN [Listener at localhost/34585] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-23 10:55:08,665 INFO [Listener at localhost/34585] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-23 10:55:08,678 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-08-23 10:55:08,767 WARN [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 286f5afc-c3f2-4f9a-8396-78cfba18066f) service to localhost/127.0.0.1:34277 2023-08-23 10:55:08,768 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data3/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,768 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data4/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,770 WARN [Listener at localhost/34585] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-08-23 10:55:08,772 INFO [Listener at localhost/34585] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-23 10:55:08,875 WARN [BP-1748601126-172.31.14.131-1692787992396 heartbeating to localhost/127.0.0.1:34277] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1748601126-172.31.14.131-1692787992396 (Datanode Uuid 0db35eca-a409-482d-977c-53ac02140840) service to localhost/127.0.0.1:34277 2023-08-23 10:55:08,876 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data1/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,876 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/bafa6cac-a1d5-c6c5-fa8a-c53402b85abc/cluster_ffcc06e0-babb-5feb-e5e3-dbb11f56847a/dfs/data/data2/current/BP-1748601126-172.31.14.131-1692787992396] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-08-23 10:55:08,899 INFO [Listener at localhost/34585] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-08-23 10:55:09,089 INFO [Listener at localhost/34585] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-08-23 10:55:09,339 INFO [Listener at localhost/34585] hbase.HBaseTestingUtility(1293): Minicluster is down